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FOREWORD 


This  research  and  development  was  performed  In  support  of  the  Navy 
Technical  Information  Presentation  Program  (NTIPP)  under  the  auspices  of 
the  Naval  Ship  Research  and  Development  Center,  Bethesda,  Maryland.  NTIPP' s 
goal  is  to  develop  a  system  of  procedures  and  equipments  designed  to  improve 
the  utility,  preparation,  revision,  storage,  distribution,  and  management 
of  technical  information  for  the  mid-1980s.  This  Center  was  tasked  with 
investigating  a  problem  fundamental  to  this  goal — assessment  of  compre¬ 
hensibility  and  usability  of  graphic  materials  in  technical  Information. 

This  report,  the  first  in  a  series,  covers  the  first  phase  of  this  work. 
It  details  the  findings  of  a  study  dealing  with  the  location  and  identifi¬ 
cation  of  information  on  technical  drawings,  outlines  initial  guidelines 
pertinent  to  graphic  comprehensibility,  and  provides  an  approach  for  further 
exploration  of  these  critical  issues. 


DONALD  F.  PARKER 
Commanding  Officer 
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SUMMARY 


Problem 


Current  graphic  practices  for  technical  manuals,  as  set  forth  In  military 
specifications,  standards,  handbooks,  and  other  publications,  are  rarely  based 
on  objective  evidence  that  they  Improve  use  and  comprehension.  Methods  based 
on  valid  data  are  not  currently  available  either  for  establishing  requirements 
for  procuring  technical  manuals  or  for  objectively  measuring  a  particular 
Illustration's  effectiveness  in  supporting  the  job  performance  of  technical 
personnel. 


The  overall  purpose  of  this  research  and  development  was  to  begin  the 
development  of  empirically  based  guidelines  and  objective  measurement 


techniques  to  Increase  the  use  of  technical  illustrations, 
end,  four  specific  objectives  were  established: 


Toward  this 


1.  To  identify  a  limited  set  of  features  that  could  be  hypothesized  as 
facilitating  or  inhibiting  the  use  of  illustrations. 

2.  To  construct  technical  illustrations  displaying  variations  of  these 
features. 


3.  To  measure  the  performance  of  Navy  technicians  extracting  selected 
information  from  these  illustrations. 

4.  To  evaluate  the  potential  of  the  present  methodology  for  future 
research. 

Approach 

The  features  selected  for  study  were  the  number,  sequence,  and  physical 
arrangement  of  callouts  or  index  numbers  used  by  technicians  in  locating  or 
identifying  parts  on  illustrations.  Starting  with  two  illustrations  from 
existing  technical  manuals — a  cross-sectional  view  and  an  exploded  view — 
these  features  were  varied  in  a  large  number  of  combinations.  The  experi¬ 
mental  tasks  of  locating  and  identifying  target  information,  given  selected 
entry  information,  were  designed  to  simulate  on-the-job  tasks  as  realistically 
as  possible  and  still  be  amenable  to  precise  measurement  in  terms  of  the  time 
required  to  complete  the  task.  Subjects  were  243  Navy  enlisted  men  in  three 
separate  ratings  who  were  familiar  to  some  extent  with  the  use  of  technical 
illustrations. 

Plndings 

1.  For  the  task  of  finding  a  part  on  an  illustration  when  given  a 
callout  number: 


a.  When  numbers  were  in  sequence,  there  was  little  difference  in 
search  time  as  the  number  of  callouts  increased  from  10  to  62. 

b.  When  numbers  were  in  random  order,  search  time  increased  by  a 
factor  of  three  or  four  as  the  number  of  callouts  increased  from  10  to  62. 

c.  Including  both  the  nomenclature  and  the  number  in  the  callout  did 
not  interfere  with  the  search  for  a  number. 

2.  For  the  task  of  finding  a  part  on  an  illustration  when  given  the 
nomenclature  of  that  part  (with  the  nomenclature  either  in  the  callout  itself 
or  in  an  accompanying  table): 

a.  For  10  callouts,  scanning  nomenclature  callouts  was  more  efficient 
than  using  a  table. 

b.  When  the  number  of  callouts  was  larger  (27  or  greater),  searching 
a  table,  even  when  not  alphabetical,  was  superior  to  scanning  nomenclature 
callouts  on  the  drawing  itself. 

c.  As  the  number  of  callouts  increased  from  10  to  62,  median  search 
time  increased  by  a  factor  of  about  six. 

d.  When  callouts  contained  both  numbers  and  nomenclature,  the  numbers 
tended  to  interfere  with  finding  a  part  name. 

3.  For  the  task  of  giving  the  nomenclature  of  a  marked  part  (using  either 
nomenclature  in  the  callout  itself  or  in  an  accompanying  table): 

a.  There  was  a  small  increase  in  search  time  from  10  to  27  callouts 
when  a  table  was  used,  and  no  increase  beyond  that. 

b.  Where  nomenclature  was  in  the  callouts,  increasing  the  number  of 
callouts  yielded  no  difference  in  response  time. 

c.  For  10  callouts,  using  a  table  or  having  numbers  in  the  callout 
itself  were  equally  efficient;  for  larger  numbers  of  callouts,  there  was  an 
advantage  to  having  nomenclature  in  the  callouts  even  when  the  number  of 
callouts  was  large  and  the  drawing  appeared  cluttered. 

4.  Findings  with  regard  to  the  physical  appearance  of  callouts  (e.g.,  the 
circling  of  a  number  versus  not  circling  it)  were  inconsistent  but  appeared 
weak  in  comparison  with  the  effects  of  sheer  number  of  callouts  and  the 
sequence  versus  random  variable. 

5.  The  general  methodology  employed  was  found  to  be  flexible  and 
relatively  efficient  and  allowed  for  variations  depending  on  the  particular 
type  of  task  in  which  the  technician  might  be  engaged. 
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Conclusions  and  Recommendations 

1.  For  part  location  by  callout  number,  always  arrange  the  callouts  in 
numerical  order. 

2.  For  part  location  by  nomenclature,  use  nomenclature  callouts  if  the 
number  of  callouts  is  10  or  less;  otherwise,  use  number  callouts  in  sequence 
keyed  to  an  alphabetical  table. 

3.  For  part  identification  (finding  the  nomenclature  when  the  location 

is  known),  use  nomenclature  callouts  even  when  the  number  of  callouts  is  large 
and  the  drawing  looks  cluttered.  There  are  no  data  on  the  upper  limit. 

4.  If  the  numbers  are  in  sequence,  then  devices  to  enhance  discrimination 
and  visual  scanning,  such  as  circling  and  aligning  the  callout  numbers,  are 
probably  unnecessary. 

5.  Zones  are  not  useful  for  locating  parts  when  a  callout  number  must 
also  be  used  for  verification. 

6.  Since  the  guidelines  differ  with  the  type  of  information-search  task, 
each  drawing  must  be  designed  for  the  specific  task  it  is  Intended  to  support. 

7.  Although  it  appears  promising  to  isolate  the  information-search 
behaviors  of  technical  manual  users  and  to  vary  the  drawing  features  that 
influence  search  speed  and  accuracy,  care  must  be  taken  in  future  studies  to 
randomize,  counterbalance,  or  measure  the  effect  of  target  location  in  the 
stimulus  materials. 

8.  Clarification  of  the  graphic  comprehension  issue  should  be  pursued 
through  empirical  studies  of  users'  information-search  behavior  and  the 
stimulus  variations  that  influence  its  effectiveness. 
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INTRODUCTION 


Problem 


Since  illustrations  may  easily  make  up  50  percent  or  more  of  a  technical 
manual,  the  effectiveness  of  graphics  in  communicating  technical  information 
is  critical.  Requirements  for  current  graphic  practices,  as  set  forth  in 
military  specifications,  standards,  handbooks,  and  other  publications,  are 
rarely  based  on  objective  evidence  that  they  improve  technical  manual  use 
and  comprehension.  Requirements  in  different  documents  sometimes  contradict 
each  other.  At  present,  there  are  no  methods  based  on  valid  data  either  for 
establishing  requirements  for  procuring  technical  manuals  or  for  objectively 
measuring  a  particular  illustration's  effectiveness  in  supporting  the  job 
performance  of  technical  personnel. 

Objective 

The  overall  objective  of  this  research  was  to  begin  development  of 
empirically  based  guidelines  and  objective  measurement  techniques  to  increase 
the  use  of  illustrations  in  technical  manuals. 

The  specific  objectives  of  this  study  were: 

1.  To  identify  a  limited  set  of  features  that  can  De  hypothesized  as 
either  facilitating  or  inhibiting  the  use  of  illustrations. 

2.  To  construct  technical  Illustrations  displaying  variations  of  these 
features . 

3.  To  measure  '.he  performance  of  Navy  technicians  in  extracting  selected 
information  from  the  illustrations. 

By  systematically  varying  a  small  number  of  well-defined  features  and 
objectively  measuring  the  effect  on  specific  tasks  simulating  actual  use,  the 
following  goals  can  be  achieved: 

1.  A  quantitative  index  of  effectiveness,  such  as  time-to-locate- 
information,  can  be  associated  with  variations  of  specific  features  as  a 
step  towards  a  more  generalized  graphic  comprehensibility  measure. 

2.  Requirements  in  specifications  and  guides  can  be  made  less  arbitrary 
by  basing  them  on  performance  data. 

Background 

Current  Status 

In  a  recent  comprehensive  survey  of  technical  manual  comprehension, 

Curran  (1977)  concluded  that: 
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Little  is  known  of  the  variables  making  up  illustrations  of 
various  kinds  and  their  relationship  to  the  comprehension 
of  that  illustration.  The  guidance  which  is  offered  technical 
illustrators  is  for  the  most  part  intuitive;  virtually,  no 
empirical  evidence  is  available. 

There  have,  of  course,  been  serious  efforts  to  control  characteristics  of 
artwork  in  technical  manuals  to  facilitate  the  transfer  of  information  to  the 
user.  At  least  two  major  approaches  can  be  cited. 

The  first  approach  is  to  implement  an  entire  philosophy  of  how  technical 
information  is  best  presented.  This  is  done  through  a  set  of  procurement 
documents  detailing  the  characteristics  of  the  manual  and  often  the  processes 
by  which  the  product  is  to  be  achieved.  Examples  are  the  Functionally 
Oriented  Maintenance  Manuals  (FOMM),  Fully  Procedural ized  Job  Performance  Aids 
(FPJPA),  NAVAIR's  Work  Package  concept,  and  the  Army's  Integrated  Technical 
Documentation  and  Training  (ITDT)  program.  Artwork  requirements,  often  quite 
detailed,  are  developed  on  the  basis  of  general  experience,  opinion,  aesthetic 
considerations,  past  practice,  reactions  against  past  practice,  and  reasoning 
from  the  overall  philosophy.  Once  the  requirements  are  set  forth  in 
procurement  documents,  technical  manuals  are  procured  in  conformance  with 
them,  and  it  is  nearly  impossible  to  investigate  the  impact  on  user 
performance  of  systematically  varying  the  more  arbitrary  aspects  of  the 
requirements . 

The  second  approach  is  represented  by  several  recent  studies  aimed  at 
relating  (1)  personnel  characteristics  such  as  test  scores,  rating,  reading 
ability,  and  experience  level,  (2)  characteristics  of  the  task  and  the  work 
environment,  and  (3)  optimum  modes  and  formats  of  data  presentation.  Recent 
work  sponsored  by  the  Navy  Technical  Information  Presentation  Program  (NTIPP) 
provides  the  beginnings  of  a  model  by  which  the  best  type  of  graphic 
presentation  can  be  selected  for  a  given  set  of  personnel,  task,  and 
environmental  factors.  This  approach  thus  far  has  been  concerned  only  with 
the  choice  of  overall  type  of  format.  Having  recommended,  for  example,  an 
exploded  view  of  an  assembly,  it  makes  no  recommendations  about  features  of 
the  drawing  itself  that  may  make  it  easy  or  difficult  to  use. 

Both  of  these  are  positive  and  important  approaches  in  that  they  focus  on 
tailoring  technical  information  to  match  the  user's  characteristics,  needs, 
and  work  environment.  The  present  NTIPP-sponsored  3tudy,  however,  is  believed 
to  be  the  first  to  employ  the  detailed  analysis  and  objective  validation 
needed  to  address  the  problem  of  establishing  requirements  for  procuring 
technical  information  or  for  objectively  measuring  the  effectiveness  of  a 
particular  illustration  in  supporting  job  performance. 

A  great  deal  of  technical  artwork  is  controlled  by  MIL-STD-100 
(Engineering  Drawing  Practices).  For  economic  reasons,  preparers  of  technical 
manuals  are  encouraged  to  use  or  adapt  engineering  drawings,  since  removing 
extraneous  material  from  existing  drawings  is  less  costly  than  producing  new 
artwork  expressly  designed  for  the  maintainer  or  operator.  Because  of  a  lack 
of  the  type  of  research  represented  by  the  present  study,  it  is  not  clear  that 
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INTRODUCTION 


Problem 


Since  illustrations  may  easily  make  up  50  percent  or  more  of  a  technical 
manual,  the  effectiveness  of  graphics  in  communicating  technical  information 
is  critical.  Requirements  for  current  graphic  practices,  as  set  forth  in 
military  specifications,  standards,  handbooks,  and  other  publications,  are 
rarely  based  on  objective  evidence  that  they  improve  technical  manual  use 
and  comprehension.  Requirements  in  different  documents  sometimes  contradict 
each  other.  At  present,  there  are  no  methods  based  on  valid  data  either  for 
establishing  requirements  for  procuring  technical  manuals  or  for  objectively 
measuring  a  particular  illustration's  effectiveness  in  supporting  the  job 
performance  of  technical  personnel. 

Objective 

The  overall  objective  of  this  research  was  to  begin  development  of 
empirically  based  guidelines  and  objective  measurement  techniques  to  increase 
the  use  of  illustrations  in  technical  manuals. 

The  specific  objectives  of  this  study  were: 

1.  To  identify  a  limited  set  of  features  that  can  be  hypothesized  as 
either  facilitating  or  inhibiting  the  use  of  illustrations. 

2.  To  construct  technical  illustrations  displayins  variations  of  these 
features. 

3.  To  measure  the  performance  of  Navy  technicians  in  extracting  selected 
Information  from  the  illustrations. 

By  systematically  varying  a  small  number  of  well-defined  features  and 
objectively  measuring  the  effect  on  specific  tasks  simulating  actual  use,  the 
following  goals  can  be  achieved: 

1.  A  quantitative  index  of  effectiveness,  such  as  time-to-locate- 
lnformation,  can  be  associated  with  variations  of  specific  features  as  a 
step  towards  a  more  generalized  graphic  comprehensibility  measure. 

2.  Requirements  in  specifications  and  guides  can  be  made  less  arbitrary 
by  basing  them  on  performance  data. 

Background 

Current  Status 


In  a  recent  comprehensive  survey  of  technical  manual  comprehension, 
Curran  (1977)  concluded  that: 
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drawings  designed  to  meet  the  needs  of  designers  and  builders  are  optimal  for 
maintainers  and  operators ,  or  to  what  degree  and  in  what  ways  they  are 
suboptimal.  Casual  examination  of  engineering  drawings  suggests  several  ways 
to  improve  such  drawings  for  maintainer  or  operator  use.  Without  empirical 
data  on  the  effects  of  using  current  engineering  drawing  practices  to  produce 
artwork  to  support  Navy  technician  performance,  it  is  impossible  to  estimate 
long  run  cost/benefit  ratios  as  an  alternative  to  immediate  cost  savings  in 
artwork  production. 

Guiding  Assumptions 

To  provide  an  orientation  for  the  development  of  specific  studies  of  the 
interaction  between  a  user  and  graphically  presented  data,  the  following 
principles  or  assumptions  were  formulated: 

1.  At  certain  points  in  the  user's  overall  sequence  of  maintenance  or 
operator  actions,  the  user  refers  to  graphic  presentations,  using  currently 
available  data  (entry  information)  to  begin  a  search  (which  may  be  simple  or 
complex)  for  needed  data  (target  information). 

2.  Certain  identifiable  characteristics  of  arrangement,  labeling, 
referencing,  and  drawing  practices  can  influence  the  efficiency  of  the  search. 
The  particular  combination  of  factors  contributing  to  an  efficient  search  will 
depend  on  the  entry  information  and  the  target  information.  Graphic 
presentations  designed  for  one  type  of  search  will  not  necessarily  be  most 
efficient  for  another  type. 

3.  In  some  cases,  the  beginning  and  end  points  of  the  information  search 
are  relatively  easy  to  identify.  Early  research  should  focus  on  such  cases, 
both  to  generate  knowledge  about  them  and  to  provide  insights  into  methodology 
for  less  obvious  and  more  complex  user  activities. 

4.  The  user  is  generally  not  aware  of  the  details  of  the  data  extraction 
process  or  of  the  factors  that  optimize  or  degrade  it.  The  user's  opinion 
about  good  and  bad  artwork  can  suggest  clues  for  further  investigation,  but 
only  direct  performance  measurement  should  be  considered  conclusive.  The  user 
may  become  aware  of  the  search  process  if  it  becomes  unusually  difficult  or 
time-consuming,  but  even  then  the  needed  improvements  may  not  be  obvious.  The 
user's  attention  is  not  focused  on  the  data  extraction  process  itself,  nor 
should  it  be;  an  efficient  search  for  data  will  be  as  short,  automatic,  and 
unintrusive  as  possible. 

5.  Even  if  an  illustrator  were  inclined  to  consider  the  search  chain  from 
entry  information  to  target  information,  the  most  effective  combination  of 
characteristics  is  not  always  obvious. 

6.  Aesthetic  considerations  alone  will  not  guarantee  optimal  usefulness 
and  may  sometimes  result  In  degrading  use.  It  may  be  necessary  to  violate 
aesthetic  principles  to  optimize  information  search.  For  example,  it  may 

be  most  beneficial  to  user  performance  to  get  a  great  deal  of  information 
on  a  single  drawing,  even  though  the  drawing  looks  cluttered. 
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7.  The  comprehension  or  usefulness  of  a  graphic  presentation  is  not  a 
property  of  the  presentation  alone,  but  depends  on  what  information  is  being 
sought  from  it  at  a  particular  moment.  It  reflects  a  relationship  between  the 
characteristics  of  the  graphic  presentation  and  the  task  being  performed  at 
that  moment.  Therefore,  an  index  of  usability  or  comprehensibility  that  does 
not  take  the  intended  use  into  account  is  seriously  deficient. 

Definition  of  a  Specific  Problem  for  Study 

Two  important  assumptions  are  (1)  that  an  illustration's  intended  use  is 
critical  in  evaluating  its  usefulness,  and  (2)  that  the  evaluation  must  be 
made  in  terms  of  measures  of  user  performance.  Because  of  these,  certain 
interesting  types  of  graphics  are  almost  automatically  excluded.  For  example, 
complex  cognitive  tasks  (e.g.,  troubleshooting)  and  the  illustrations  that 
support  them  (e.g.,  schematics,  block  diagrams)  were  not  considered  amenable 
at  this  time  to  a  fine-grained  objective  study.  It  is  expected  that  the  type 
of  research  represented  by  the  present  study  will  suggest  ways  of  objectively 
studying  these  other  important  graphic  types  and  the  behaviors  associated  with 
them. 

Location  and  identification  of  parts,  however,  were  judged  to  be  highly 
suitable  user  activities  for  the  present  study  for  the  following  reasons: 

1.  They  are  common  activities  among  technical  manual  users. 

2.  They  are  supported  by  a  number  of  different  types  of  drawings;  for 
example,  isometric  drawings,  cross-sectional  views,  exploded  views, 
circuit-board  drawings,  and  control-panel  drawings. 

3.  Elements  of  the  drawings  that  are  intended  to  support  this  type  of 
search,  such  as  callouts  and  zones,  are  obvious.  Varying  these  elements  for 
experimental  purposes  is  not  difficult. 

4.  Advice  and  requirements  relating  to  these  elements  are  not  always 
consistent,  and  sometimes  appear  to  derive  from  considerations  of  aesthetics 
and  contractor  convenience  rather  than  effectiveness  and  user  convenience. 

5.  Apparent  violations  of  human  factors  considerations  and  even  common 
sense  are  not  difficult  to  find  in  recent  technical  manuals. 

6.  The  user's  information  search  task  can  be  easily  and  realistically 
simulated  with  experimental  controls. 

Even  though  the  present  study  has  been  limited  to  part  location  and 
identification,  the  scope  of  questions  that  might  be  asked  is  still  very  broad 
and  includes  the  following: 

1.  Should  numbered  callouts  be  in  sequence?  What  price  in  efficiency  is 
paid  if  they  are  not?  Are  there  cases  where  numerical  sequence  is 
unimportant? 
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2.  What  is  the  maximum  number  of  callouts  that  should  appear  on  a 
drawing?  Do  number  callouts  and  nomenclature  callouts  differ  in  this  respect? 

3.  What  is  the  trade-off  between  having  nomenclature  callouts  on  the 
drawing  versus  putting  the  nomenclature  in  a  table  keyed  to  number  callouts? 

4.  When  reference  designators  are  used  as  callouts,  are  they  less 
discriminable  than  nomenclature  or  numbers,  and  therefore  harder  for  a  user  to 
find  quickly? 

5.  Do  graphic  devices  such  as  circling  the  numbers  or  using  large,  bold 
type  help  the  user  to  scan  more  effectively? 

6.  Should  leaders  (the  line  connecting  the  callout  to  the  part)  or  arrows 
be  short  so  that  the  callout  is  close  to  the  part  it  identifies,  or  is 
scanning  aided  if  the  leaders  are  extended  so  that  the  callouts  are  arranged 
in  straight  lines? 

7.  If  alphanumeric  zones  are  used,  what  zone  size  is  most  effective? 

8.  Should  the  zone  reference  be  keyed  to  the  center  of  the  part,  the 
callout,  or  the  arrowhead? 

9.  Zone  designators  on  engineering  drawings  use  the  lower  right  corner  as 
the  origin  and  run  backwards  from  the  normal  reading  direction  (right  to  left 
and  bottom  to  top).  Does  this  degrade  search  performance? 

There  are,  of  course,  additional  questions  concerning  the  interaction  of 
the  various  factors  with  each  other  and  with  the  type  of  search  being 
performed.  Some  of  the  above  questions  were  addressed  in  the  present  study. 

To  limit  the  scope  of  the  present  study,  however,  reference  designators  were 
not  studied,  and  zones  were  represented  only  minimally  to  collect  information 
for  the  design  of  a  future  study. 

Three  common  types  of  information  search  related  to  part  location  and 
identification  were  simulated  in  this  study: 

1.  A  part  is  cited  by  callout  number  in  a  procedure,  explanation,  or 
description:  Find  the  part  in  the  drawing. 

2.  A  part  is  cited  by  nomenclature  in  a  procedure,  explanation,  or 
description:  Find  the  part  in  the  drawing. 

3.  A  part  location  in  a  drawing  is  known  (for  instance,  by  recognizing 
its  physical  appearance):  Find  its  nomenclature. 

Examples  of  the  Problem  Today 


The  proper  use  of  callouts  and  zones  to  aid  the  user  may  seem  so  simple 
and  intuitive  that  some  readers  may  doubt  that  a  problem  really  exists.  It  is 
appropriate,  therefore,  to  present  examples  of  practices  currently  prescribed 


and  employed  in  technical  illustrations.  First,  examples  are  presented  from 
textbooks,  manuals,  and  military  specifications  and  handbooks  whose  function 
is  to  guide  in  the  production  of  technical  artwork.  Second,  examples  are 
taken  from  technical  documents  already  in  use  in  the  Navy.  It  should  be  kept 
in  mind  that  not  all  of  these  examples  represent  practices  that  always  degrade 
performance;  one  of  the  major  points  in  the  present  approach  is  that  features 
contributing  to  usability  will  differ  depending  on  how  the  illustration  is  to 
be  used. 

Figure  1,  an  illustration  from  a  mechanical  drawing  textbook  (Giesecke, 
Mitchell,  Spencer,  &  Hill,  1974,  p.  419),  is  intended  as  an  example  of  a 
correctly  produced  cross-sectional  view  and  is  a  type  of  engineering  drawing 
frequently  reproduced  "as  is"  in  maintenance  manuals.  Its  lack  of  numerical 
sequence  in  the  placement  of  callouts  is  common  in  such  drawings.  An 
intuitive  recommendation,  corroborated  by  this  experiment,  is  that  finding 
information  by  means  of  callouts  would  be  much  more  efficient  if  the  callouts 
were  in  numerical  sequence. 

Figure  2,  found  in  another  illustrating  guidebook  (French,  Svensen, 

Helsel,  &  Urbanick,  1974,  p.  494),  is  perhaps  an  even  more  severe  violation  of 
the  same  principle.  Depending  on  the  particular  callout  number  selected,  it 
may  take  a  frustratingly  long  time  to  locate  a  particular  number  on  the 
drawing.  This  is  a  good  example  of  the  need  to  consider  the  intended  use  of 
the  drawing.  If  the  user  enters  the  drawing  with  the  name  of  the  part  and  his 
goal  is  to  find  the  location,  then  the  parts  list  should  probably  be  in 
alphabetical  order  and  the  callouts  should  be  in  numerical  order.  This  task 
is  made  maximally  difficult  by  the  arrangement  shown,  because  a  random  search 
is  necessary  for  both  the  part  name  and  the  callout.  If  the  task  is  to  find 
the  name  of  a  part  given  its  location  or  physical  appearance,  then  the  drawing 
is  acceptable,  since  the  callouts  do  not  have  to  be  searched  and  ordering  the 
parts  list  by  callout  number  is  helpful. 

Civilian  illustrating  guides  are  not  alone  in  providing  questionable 
recommendations.  A  new  DoD  publication.  Technical  Manual  Writing  Handbook 
(1977),  contains  the  "wrong-right”  examples  shown  here  as  Figure  3.  The 
message  of  these  examples  is  a  warning  against  extending  leaders  (the  lines 
connecting  the  numbers  or  nomenclatures  to  the  parts)  to  obtain  a  simpler 
scanning  pattern.  Although  there  is  no  evidence  that  user  performance  is 
aided  by  the  "right"  approach  or  degraded  by  the  "wrong"  one,  no  mention  is 
made  in  the  handbook  of  arranging  callouts  in  sequence,  which  turns  out  to  be 
a  much  more  powerful  variable  in  increasing  scanning  efficiency  than  length  of 
leaders.  Such  misplaced  emphasis  apparently  occurs  because  aesthetic 
considerations  prevail  in  the  absence  of  hard  evidence  about  the  factors  that 
actually  enhance  usability. 

Figure  4  is  taken  from  a  writing  guide  (Post  &  Price,  1974)  that  was 
prepared  under  contract  for  the  Naval  Sea  Systems  Command  but  was  never  used 
officially  for  TM  acquisition.  Because  of  the  small  number  of  callouts  in  the 
example,  the  lack  of  numerical  sequence  would  probably  cause  little 
deterioration  in  performance,  but  a  principle  is  illustrated  that  would  be 
detrimental  when  applied  in  other  circumstances.  For  example,  a  technical 
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(Excerpted -from  Giesecke,  Mitchell,  Spencer,  and  Hill,  "Technical  Drawing," 
Copyright  1974,  Macmillan  Publishing  Co.,  Inc.) 

Figure  1.  Cross-sectional  view  from  a  mechanical  drawing  textbook  with 
callouts  numbered  nonsequentially . 
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1  Radome 

2.  Forward  fuselage 

3.  Pilot  seat 

4  Windshield 

5.  Forward  canopy 

6.  Radar  operation  seat 
7  Alt  canopy 

10  Noae  landing  gear  door,  forward 

1 1  Nose  landing  gear  door,  aft 

12.  Hydraulic  compartment 
acceaa  door 

13.  Note  landing  gear  shock  strut 


14.  Center  fuselage  29. 

16.  Fuat  tank  door  30. 

17.  Engine  access  door  31. 

18.  Engine  access  door  32. 

19.  Engine  access  door  33. 

20  Engine  access  door  34. 

21.  Auxiliary  engine  air  door  39. 

23.  Arresting  hook  36. 

24.  Aft  fuselage  37. 

25.  Tail  cone  36. 

26.  Rudder  39. 

28.  Stabilator  40. 


Cantor  section  wing 

Leading  edge  flap 

Main  landing  gear  strut  door 

Main  landing  gear  inboard  door 

Inboard  spoiler 

Outboard  spoiler 

Flap 

Aileron 

Spaad  brake 

Main  landing  gear  shock  strut 
Outer  wing 

Loading  edge  nap.  outboard 


Leading  edge  flap,  inboard 
Missile  rack 
Bomb  rack 
Missile  pylon 

External  center-line  fuel  tank 
Oata  link  access  door 
External  wing  fuel  tank 
External  wing  fuel  tank  pylon 
Landing  gear  door,  outboard 
Boom  IFR  receptacle  aocasa  door 
Fuel  ce>  acceaa  door 
Fuel  oeN  accaea  door 


41. 

44 

45 

46 

47 
49. 

53 

54 

55 

56 

57 
62. 


(Excerpted  from  French,  Svensen,  Helsel,  and  Urbanick,  "Mechanical  Drawing." 
Copyright  1948,  McGraw-Hill,  Inc.) 

Figure  2.  An  exploded  view  from  a  mechanical  drawing  textbook  in  which 
both  callout  numbers  and  table  items  tend  to  increase  the 
difficulty  of  its  use. 


1  Radome 

2  Forward  fuselage 
3.  Pilot  $«at 

4  Windshield 

5  Forward  canopy 

6.  Radar  operation  seat 
7  Alt  canopy 

10  Nose  landing  gear  door,  lorward 
tl  Nose  landing  gear  door,  alt 
12  Hydraulic  compartment 
acceea  door 

13.  Nose  landing  gear  shock  strut 


14.  Center  fuselage  29. 

16.  Fuel  tank  door  30. 

17  Engine  access  door  31. 

18.  Engine  access  door  32. 

19.  Engine  access  door  33. 

20  Engine  access  door  34. 

21  Auxiliary  engine  air  door  35. 

23.  Arresting  hook  36. 

24.  Alt  fuselage  37. 

25.  Tail  cone  36. 

26.  Rudder  39. 

29.  Stabilator  40. 


Center  section  wing 

Leading  edge  flap 

Main  landing  gear  strut  door 

Main  landing  gear  inboard  door 

Inboard  spoiler 

Outboard  spoiler 

Flap 

Aileron 

Speed  brake 

Main  landing  gear  shock  strut 
Outer  wing 

Loading  edge  flap,  outboard 


41.  Leading  edge  flap,  inboard 

44.  Missile  rack 

45.  Bomb  rack 

46.  Missile  pylon 

47.  External  center-line  fuel  tank 
49.  Oata  link  access  door 

53.  External  wing  fuel  tank 

54.  External  wing  fuel  tank  pylon 

55.  Landing  gear  door,  outboard 

56.  Boom  IFR  receptacle  access  door 

57.  Fuel  call  acceea  door 
62.  Fuel  call  acceea  door 


(Excerpted  from  French,  Svensen,  Helsel,  and  Urbanick,  "Mechanical  Drawing." 
Copyright  1948,  McGraw-Hill,  Inc.) 

Figure  2.  An  exploded  view  from  a  mechanical  drawing  textbook  in  which 
both  callout  numbers  and  table  items  tend  to  increase  the 
difficulty  of  its  use. 


MIL-HD3K-63038-1 


When  alining  numbers  with  each  other,  don’t  overdo  it.  The  example  below  left  demonstrates  a 
misplaced  effort  at  good  paste-up  layout.  Keeping  the  numbers  even  requires  arrows  much  too 
long  The  example  below  right  demonstrates  good  paste-up  layout 


Figure  3.  Example  of  a  questionable  recommendation  in  a  military  style 
guide. 
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illustrator  (particularly  a  novice,  who  is  most  likely  to  be  using  such 
guides),  in  following  the  manual's  recommendations,  might  be  very  careful  to 
avoid  the  use  of  long  leaders,  which  may  or  may  not  be  detrimental  to  the 
user,  and  pay  no  attention  to  the  ordering  of  callouts,  which  seriously 
affects  search  behavior. 

A  military  publication  that  also  offers  acceptable  examples  of  "good"  and 
"bad"  graphic  practices  (Figures  5a  and  5b)  is  NAVAIR's  Technical  Manual 
Preparation  Guide  (1975).  Figure  5b  is  an  interesting  instance  in  which  rigid 
adherence  to  a  clockwise  sequence  of  numbers  might  best  be  abandoned.  Part  27 
is  actually  joined  with  part  13,  and  probably  should  follow  part  13  in 
sequence.  Therefore,  rather  than  slavishly  following  the  "clockwise"  rule, 
the  callouts  can  be  kept  in  sequence  and  at  the  same  time  show  the  actual 
interrelationship  of  the  parts  along  the  center  (axis)  line,  as  shown  in 
Figure  5c. 

The  technique  of  arranging  callouts  in  sequence  is  not  a  new  development. 
It  is,  in  fact,  prescribed  by  Military  Specification  MIL-M-38784A  (1975), 
which  contains  DoD-wide  requirements  for  technical  manual  style  and  format. 

The  specification  states,  "Index  (callout]  numbers  for  each  separate  figure 
shall  start  with  Arabic  number  1  and  continue  consecutively.  Sequence  shall 
be  from  top  to  bottom  or  clockwise,  when  possible."  This  requirement  is 
weakened,  however,  when  the  specification  states,  "Item  numbers  on  exploded 
views  used  to  show  assembly/disassembly  shall  be  in  disassembly  order."  In 
cases  where  the  callout  numbers  are  searched,  even  when  the  purpose  of  the 
drawing  is  to  aid  assembly  or  disassembly,  sequential  ordering  would  appear  to 
be  in  the  best  interests  of  the  user.  The  only  exception  might  be  in  simple 
drawings  with  few  callouts,  where  the  callout  number  sequence  is  the  only 
device  used  to  indicate  assembly  or  disassembly  order. 

MIL-M-38784A  states  that  leader  lines  "shall  be  uniform,  short,  and 
straight  as  possible."  However,  this  "cosmetic"  characteristic  was  tested  in 
the  present  experiment  (the  condition  referred  to  herein  as  "extending")  and 
showed  no  conclusive  effect  on  performance.  Again,  aesthetic  considerations, 
rather  than  knowledge  of  effects  on  performance,  seem  to  be  the  basis  of 
requirements  in  the  military  specification. 

Another  military  specification,  MIL-M-15071G,  deals  with  content 
requirements  for  Navy  technical  manuals.  Despite  the  fact  that  it  requires 
conformance  with  MIL-M-38784A,  its  examples  do  not  conform,  as  indicated  by 
Figure  6.  It  states,  as  do  other  sources,  that  parts  in  an  exploded  view 
shall  be  arranged  in  "correct  relative  disassembly  position."  It  says  nothing 
directly  about  callout  arrangement,  but  the  implication  of  Figure  6  is  clear. 
No  consideration  is  given  to  aiding  the  user's  search. 

A  major  question  addressed  in  this  experiment  is  the  number  of  callouts 
that  may  be  used  on  a  given  illustration  and  the  interactions  of  other  factors 
with  that  number.  Post  and  Price  (1974),  under  the  heading  "Pictorials 
Practices,"  recommended  that  callouts  be  limited  to  seven  or  less.  No 
distinction  was  made  between  nomenclature  and  number  callouts.  This  surely 
does  not  mean,  for  example,  that  on  an  11  x  17  inch  drawing  there  should  be  no 
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more  than  seven  number  callouts.  What  it  does  mean,  however,  is  not  clear. 

It  seems  to  recommend  that  no  more  than  seven  callouts  appear  in  a  grid  of 
some  unspecified  size.  A  similar  warning  appears  in  the  NAVAIR  guide,  which 
states  that  there  should  be  no  more  than  five  number  callouts  in  any 
1-square-inch  area.  It  further  specifies  that  "...the  average  maximum  number 
of  callouts  within  a  7  x  9  inch  area  should  be  70."  This  requirement  is  badly 
stated,  since  it  would  allow  dense  packing  of  callouts  in  some  drawings  as 
long  as  there  were  other  drawings  with  low  callout  density  to  bring  down  the 
average.  Whatever  the  origin  and  validity  of  these  numerical  limits,  the 
authors  of  these  documents  seem  to  believe  that  they  are  better  than  no 
guidelines  at  all.  Research  should  be  directed  at  establishing  specific 
limits  based  on  user  performance.  These  limits  would  probably  differ 
depending  on  whether  callouts  were  numbers,  nomenclature,  reference 
designators,  or  combinations  of  these  types.  At  this  time,  no  one  really 
knows  the  maximum  density  of  callouts  with  which  a  user  can  easily  contend. 

Very  little  searching  is  required  to  locate  violations  of  the  sort 
discussed  above  in  virtually  any  type  of  Navy  technical  manual  for  any  type  of 
system  or  equipment.  Appendix  A  contains  six  examples  of  the  stimuli  actually 
used  in  this  experiment,  all  of  which  are  variations  of  two  actual  technical 
manual  illustrations,  both  of  which  were  considered  unacceptable  for  various 
reasons.  Appendix  B  contains  seven  additional  examples  (reproduced  actual 
size)  from  different  kinds  of  Navy  manuals. 

Figure  B-l  is  included  for  two  reasons.  First,  its  callout  arrangement, 
while  not  as  difficult  to  use  as  some  others,  is  not  in  sequence,  and  callout 
numbers  jump  from  one  drawing  in  the  figure  to  another  and  back  again.  If 
there  are  several  drawings  on  a  page,  then  callouts  should  probably  be 
arranged  in  numerical  order  in  each  of  the  drawings,  even  if  the  drawings  are 
intimately  interrelated.  Second,  Figure  B-l  is  quite  similar  to  the  original 
of  the  cross-sectional  drawing  used  as  an  experimental  stimulus  in  the  present 
study,  including  the  hand  lettering  of  numbers  and  nomenclature.  With  as  few 
callouts  as  are  found  on  this  drawing,  the  use  of  nomenclature  in  the  callouts 
seems  appropriate.  With  relatively  few  callouts,  if  the  user's  task  is  to 
identify  a  part,  then  it  is  probably  more  efficient  if  the  nomenclature  is  in 
the  callout  than  if  it  is  in  an  accompanying  table. 

Figure  B-2  is  from  a  different  section  of  the  same  manual  as  Figure  B-l. 

It  is  legitimate  to  use  such  drawings,  according  to  MIL-M-38784A,  "provided 
they  meet  the  reproducible  [sic]  requirements  herein.  Borders,  title  blocks, 
manufacturer's  notes  and  irrelevant  material  shall  be  removed."  In  the  case 
of  Figure  B-2  there  is  severe  "information  compression"  caused  by  reducing  the 
size  of  the  original  drawing,  there  is  no  organized  callout  sequence,  and  the 
manufacturer's  detailed  information  was  not  removed.  Any  one  of  these  factors 
should  have  been  cause  for  rejection  of  the  drawing,  but  this  illustration  and 
many  others  similar  to  it  were  accepted,  at  least  for  the  preliminary  manual. 

Figure  B-3  is  taken  from  a  sonar  systems  manual  dealing  with  towed-array 
hoists.  It  is  included  to  illustrate  a  common  problem,  that  of  undersized 
print  in  the  callouts.  Another  problem  in  Figure  B-3  is  that  the  leader  lines 
are  drawn  in  such  a  way  that  they  might  be  confused  with  the  lines  of  the 
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drawing.  Again,  there  is  a  lack  of  systematic  organization  of  the  numbers  on 
the  drawing.  It  is  exceptionally  difficult  to  get  started  on  this  drawing, 
that  is,  to  locate  callout  number  1. 

Figure  B-4  is  taken  from  an  ordnance  publication.  The  drawing  includes 
essentially  three  different  kinds  of  callouts.  Some  callout  numbers  are 
circled,  some  are  not,  and  some  are  "stacked,"  which  many  sources  discourage 
(with  no  supporting  empirical  evidence).  Common  sense  alone  would  suggest 
greater  consistency  in  the  physical  characteristics  of  callouts  on  a  drawing. 

Figure  B-5  is  taken  from  an  engineering  system  technical  manual  (NAVSHIPS 
0951-010-0010).  It  suffers  from  the  common  faults  of  failing  to  order 
callouts  systematically  and  of  requiring  the  reader  to  jump  back  and  forth 
between  the  figure's  drawings  to  follow  the  numerical  sequence. 

The  fact  that  Figures  B-l  through  B-5  were  taken  from  different  types  of 
Navy  manuals  indicates  the  widespread  nature  of  the  problems  being  addressed 
in  this  study  and,  consequently,  the  need  for  corrective  action. 

The  Naval  Air  Systems  Command  has  generally  been  much  more  consistent  than 
other  SYSCOMs,  from  the  standpoint  of  aiding  the  user,  in  the  illustration  of 
its  manuals.  This  tendency  has  been  advanced  by  its  "Work  Package"  concept, 
which  makes  use  of  a  larger  number  of  small,  special-purpose  illustrations 
rather  than  a  smaller  number  of  large,  complex,  general-purpose  ones.  (The 
Army's  Integrated  Technical  Documentation  and  Training  (ITDT)  program  is 
fostering  the  same  trend.)  Even  NAVAIR  technical  information,  however,  is  not 
totally  free  of  the  types  of  problems  discussed  above.  Figure  B-6  illustrates 
the  random  order  of  callouts  in  a  work  package.  The  accompanying  procedural 
steps  illustrate  what  may  be  the  cause  of  this  problem  in  many  cases:  The 
callouts  are  ordered  as  they  are  referred  to  in  the  procedure.  This  is  common 
practice  in  both  military  and  commercial  drawings  that  support  procedural 
steps,  but  it  does  the  user  no  good  to  have  the  parts  so  numbered  because  the 
task  does  not  involve  searching  the  steps  for  callout  numbers.  Upon 
encountering  a  callout  number  in  a  procedural  step,  the  user  must  search  the 
drawing,  which  is  where  the  numbers  ought  to  be  in  sequence.  Performance 
would  not  be  degraded  in  the  least  if  the  callout  numbers  were  out  of  order  in 
the  procedural  steps,  but  would  be  improved  significantly  if  they  were  in 
order  on  the  drawing. 

Figure  B-7,  from  NAVSHIPS  91921(A),  Instruction  Book  for  Radar  Set 
AN/SPS-10,  was  considered  so  unsuitable  for  maintenance  use  that  it  was 
rejected  as  a  stimulus  for  the  present  experiment.  In  particular,  note  the 
two  1-inch  squares  drawn  on  the  illustration.  As  mentioned  earlier,  various 
sources  advise  limiting  callouts  to  about  five  to  seven  per  1-square-inch 
area,  but  one  of  the  squares  on  Figure  B-7  has  12  and  the  other  has  11. 
Requirements  to  limit  callout  density  are  obviously  needed,  but  they  should  be 
based  on  performance  data,  particularly  since  research  to  generate  such  data 
is  neither  difficult  nor  expensive. 

The  matter  of  zones  (coordinates),  although  receiving  minimal  attention  in 
the  present  study,  provides  interesting  examples  of  the  dearth  of  good 
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information  on  how  to  aid  the  user  in  locating  parts.  For  example, 
MIL-M-15071G  requires  zoning  as  follows: 

Diagrams  shall  be  divided  into  equally  spaced  horizontal  zones 
(ordinates)  designated  A,  B,  etc.,  from  bottom  to  top  along 
the  outside  left  and  right  borders.  Diagrams  shall  be  divided 
into  equally  spaced  vertical  zones  (abscissa)  designated  1,  2, 

3,  etc.,  from  right  to  left  along  the  outside  top  and  bottom 
borders.  The  zone  size  shall  be  as  needed  to  clearly  locate 
referenced  points. 

Note  that  the  required  order  of  numbers  and  letters  along  the  axes  is  the 
reverse  of  normal  reading.  Engineering  drawings  have  zones  arranged  in  this 
fashion,  and  the  requirement  in  MIL-M-15071G  is  apparently  intended  to 
eliminate  the  need  to  redraw  the  axes.  The  benefit  is  to  the  producer  of  the 
drawing,  not  to  the  user .  Zone  size  is  left  largely  to  the  discretion  of  the 
illustrator.  It  should  not  be  difficult  to  develop  data  for  a  functional 
relationship  between  part  density,  zone  size,  and  search  performance  to 
formulate  intelligent  requirements. 

WS-10759  (NAVORD  Specification/Purchase  Description  for  Ordnance 
Publications),  a  specification  frequently  invoked  by  NAVSEA  for  weapon  system 
manuals,  requires  zones  as  follows: 

Diagrams  shall  be  divided  into  three  horizontal  zones 
(ordinates)  with  the  lower  zone  designated  A,  the  middle  zone 
designated  B,  and  the  upper  zone  designated  C  outside  the  left 
and  right  borders.  Diagrams  shall  be  divided  into  vertical 
zones  (abscissa)  4-3/4  inches  wide  final  size,  designated  1, 

2,  3,  etc.,  from  right  to  left  outside  the  top  and  bottom 
border. 

The  zones  are  again  "backwards,"  but  this  time  they  are  required  to  be  quite 
large . 

As  a  final  example  of  the  range  of  variation.  Figure  7  is  taken  from  a 
Functionally  Oriented  Maintenance  Manual  (FOMM),  NAVSEA  0967-LP-603-8030.  FOMM 
manuals  are  based  on  an  explicit  user-oriented  approach.  In  this  case  the 
effort  to  aid  the  user  resulted  in  "overkill,"  thus  increasing  the  difficulty 
of  the  user's  search.  The  double  indexing  of  zones  to  achieve  0.2-inch 
resolution  results  in  a  cumbersome  zone  designation  system  (e.g.,  D.E/7.5). 
First,  this  places  an  unnecessary  load  on  the  immediate  memory  of  the  user  and 
leads  to  an  increased  chance  of  reading  errors.  Second,  this  degree  of 
precision  is  not  necessary  to  locate  the  parts  quickly.  Third,  users  are 
unable  to  track  horizontally  or  vertically  on  the  page  to  this  degree  of 
precision  without  an  overlay  grid  or  a  straightedge.  The  result  is  that  users 
quickly  learn  to  read  and  remember  only  the  major  coordinates  (e.g.,  D-7)  and 
to  locate  the  part  accordingly.  Users  should  not  have  to  learn  to  ignore 
information  that  was  put  there  to  help  them. 
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Figure  7.  Technical  manual  illustration  with  fine-grained  zones. 
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It  is  clear  from  these  examples  that  there  is  little  empirically  based 
information  to  guide  illustrators  in  enhancing  user  performance,  and  intuition 
is  an  unreliable  guide.  Even  conscious  efforts  to  aid  the  user  sometimes 
misfire.  The  necessary  data  would  often  not  be  difficult  to  obtain  from  the 
standpoint  of  methodology,  but  would  require  a  large  program  of  painstaking 
research.  The  present  study  is  an  effort  to  inspire  such  a  program. 
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METHOD 


The  general  approach  was  to  identify  tasks  that  closely  simulate  what  a 
technician  does  when  he  is  using  a  graphic  presentation  to  get  needed 
information.  These  tasks  were  restricted  to  location  and  identification  of 
parts  by  use  of  callouts  with  and  without  parts  lists,  and,  to  a  minor  extent, 
zones.  As  described  below,  variations  in  number,  content,  and  arrangement  of 
callouts  were  incorporated  into  drawings,  were  presented  to  subjects  using 
five  different  types  of  task  instructions,  and  were  evaluated  according  to  the 
time  required  for  the  specified  information  search. 

Illustration  Types 

Stimulus  variations  were  incorporated  into  two  basic  drawings  taken  from 
Navy  technical  manuals:  a  cross-sectional  view  (C/S)  of  an  electric  motor, 
taken  from  the  AN/SPS-58  Radar  Maintenance  Manual  (NAVSHIPS  0967-320-4040), 
and  an  exploded  view  (E/V)  of  a  chart  drive.  These  are  typical  of  drawings 
found  in  all  types  of  Navy  technical  manuals.  The  originals  were  modified  to 
produce  the  experimental  variations,  examples  of  which  are  reproduced  in 
Appendix  A.  All  types  of  variation  are  illustrated  in  these  examples.  The 
original  drawings  appear  as  Figures  A-2  and  A-6,  except  that  Figure  A-2  was 
hand  lettered  in  its  original  form. 

Variations  of  the  C/S  are  not  directly  comparable  to  those  of  the  E/V.  In 
particular,  the  C/S  as  originally  drawn  had  nomenclature  callouts,  which  were 
retained  in  some  of  the  variations.  Because  of  the  placement  of  callouts  on 
the  E/V,  nomenclature  callouts  were  completely  impractical. 

Number  of  Callouts 

The  number  of  callouts  on  a  drawing  was  varied  in  four  steps:  10,  27,  44, 
and  62.  This  variable  was  applied  in  identical  fashion  to  both  the  C/S  and 
the  E/V. 

Arrangement  and  Content  of  Callouts 

There  were  13  variations  in  the  arrangement  and  content  of  callouts.  The 
variations  were  based  on  two  types  of  drawings  (cross-sectional  and  exploded 
views),  three  methods  of  labeling  (number,  nomenclature,  or  both),  and  three 
treatment  variables  (callout  order,  number  circling,  and  leader  extension). 
Each  of  the  treatment  variables  consisted  of  two  levels: 

1.  Order:  Callouts  could  be  arranged  sequentially  (SEQ)  or  at  random 
(RAN). 

2.  Circling:  Numbers  could  be  circled  (C)  to  distinguish  them  from  other 
elements  of  the  drawing,  or  could  be  not  circled  (NC). 

3.  Extension:  Callout  leaders  could  be  extended  (E)  to  the  drawing's 
edge  to  line  up  the  callouts  and  make  scanning  easier,  or  have  nonextended 
(NE)  leaders  of  more  uniform  length.  In  the  E  conditions,  callouts  that  could 
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not  be  brought  to  the  edge  due  to  space  limitations  were  lined  up  diagonally 
near  the  center  of  the  drawing. 

Of  the  13  variations,  5  were  applied  to  the  C/S: 

1.  Nomenclature  only  (NOMEN). 

2.  Numbers  in  sequence  (NUM-SEQ). 

3.  Numbers  in  random  order  (NUM-RAN). 

4.  Nomenclature  with  numbers  in  sequence  (N/N-SEQ). 

5.  Nomenclature  with  numbers  in  random  order  (N/N-RAN). 

The  remaining  eight  variations  (illustrated  in  Figures  A-l,  A-3,  A-4,  and 
A-5)  used  only  number  callouts: 

1.  Sequential,  circled,  extended  (SEQ/C/E). 

2.  Sequential,  circled,  not  extended  (SEQ/C/NE). 

3.  Sequential,  not  circled,  extended  (SEQ/NC/E). 

4.  Sequential,  not  circled,  not  extended  (SEQ/NC/NE). 

5.  Random,  circled,  extended  (RAN/C/E). 

6.  Random,  circled,  not  extended  (RAN/C/NE). 

7.  Random,  not  circled,  extended  (RAN/NC/E). 

8.  Random,  not  circled,  not  extended  (RAN/NC/NE). 

Subject  Tasks 

Five  subject  tasks  were  defined  that  simulate  user  behavior  in  the  work 
environment.  Each  is  best  described  by  its  corresponding  instruction  to  the 
subject : 

1.  "Point  to  the  part  with  callout  number  X."  (Callout  appears  in  a 
procedure  or  equipment  description  referencing  a  figure.) 

2.  "Point  to  the  part  called  Y."  (Nomenclature  appears  in  a  procedure  or 
description  referencing  a  figure.) 

3.  "Tell  me  the  nomenclature  of  the  part  marked  in  red."  (Physical 
appearance  or  location  of  the  part  is  known,  and  the  nomenclature  is  sought.) 

4.  "Use  the  zone  system  to  point  to  the  part  with  callout  number  X." 
(Same  situation  as  in  Task  1.) 
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5.  "Use  the  zone  system  to  point  to  the  part  called  Y."  (Same  situation 
as  Task  2.) 

The  zone  system  refers  to  the  alphanumeric  coordinates  appearing  on  the 
exploded  view.  Given  the  callout  number  or  nomenclature,  the  subject  must  use 
a  table  to  find  the  coordinates  of  the  part. 

Test  Items 

Figure  8  shows  all  possible  combinations  of  task  and  stimulus  variations. 
Each  of  these  combinations  defines  a  single  test  item  that  could  potentially 
be  administered  to  experimental  subjects.  A  subset  of  160  of  these,  indicated 
in  Figure  8  by  diagonal  lines,  was  actually  used  in  the  experiment. 

Task  1  NOMEN  would  have  necessitated  the  use  of  a  table,  and  did  not 
appear  to  lend  itself  to  sufficiently  interesting  comparisons.  For  Tasks  4 
and  5,  which  involved  the  use  of  the  zone  system,  the  intent  was  to  develop 
some  preliminary  information  rather  than  to  be  exhaustive.  It  was  decided 
that  the  zone  system  would  be  useful  only  in  situations  where  there  was  a 
relatively  large  number  of  callouts  in  random  order,  so  only  these  test  items 
were  used. 

Before  each  test  item,  the  subject  was  given  (1)  background  information  to 
simulate  what  a  user  would  ordinarily  know  about  a  drawing,  (2)  entry 
information,  and  (3)  his  task.  Thus,  in  a  typical  test  item,  the  subject 
would  be  told,  "The  next  drawing  is  the  exploded  view  with  callouts  that  have 
numbers  out  of  order.  Point  to  the  part  with  callout  number  28.  Go."  The 
subject  would  then  turn  to  the  drawing  and  find  the  required  part. 

Selection  of  Target  Information 

For  each  test  item,  a  part  was  selected  whose  location  or  identity  was  the 
target  of  the  information  search.  Targets  were  selected  with  three  primary 
criteria : 


1.  Responding  to  a  test  item  should  not  aid  the  subject  on  a  later  item. 

2.  Targets  were  selected  equally  from  all  areas  of  the  drawings. 

3.  Items  differing  only  by  the  number  of  callouts  were  assigned  targets 
that  were  not  identical  but  that,  to  the  extent  possible,  were  in  the  same 
area  of  the  drawing. 

Accompanying  Tables 

In  Tasks  2,  3,  4,  and  5,  the  subject  often  required  information  that  was 
not  part  of  the  callout.  Tables  (parts  lists)  were  used  to  supply  the  missing 
information.  In  Tasks  2  and  3  the  table  provided  the  bridge  between  callout 
number  and  nomenclature,  and  in  Tasks  4  and  5  the  table  provided  zone 
information.  All  tables  were  in  the  same  format.  The  number  of  items  in  each 
table  corresponded  to  the  number  of  callouts  on  the  drawing,  and  the  items 
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Figure  8.  Diagram  of  experimental  design:  stimulus-task  combinations 
used  in  study  are  cross-hatched;  T  indicates  use  of  parts 
list  table  required. 
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were  listed  in  callout  number  sequence.  The  tables  were  therefore  efficient 
when  entered  by  callout  number,  and  less  efficient  when  entered  by 
nomenclature.  No  task  required  the  subject  to  enter  the  table  by  zone 
designation.  Test  items  that  required  the  use  of  a  table  are  indicated  in 
Figure  8. 

Groups 

Preliminary  trials  using  the  stimulus  materials  indicated  that  to 
administer  all  160  test  items  in  one  session  would  have  been  too  fatiguing  for 
the  subject.  Therefore,  the  160  test  items  were  divided  into  four  groups  of 
40,  with  the  various  levels  of  each  independent  variable  represented  equally 
in  each  group.  One  group  of  40  test  items  was  administered  to  each  subject. 

Other  Materials 


In  addition  to  the  drawings,  which  were  bound  in  scrambled  order  for  the 
subjects'  use,  there  was  a  corresponding  set  of  item  description  sheets  to  be 
read  to  the  subjects  in  connection  with  each  drawing.  Before  the  test  items 
were  administered,  each  subject  completed  a  data  sheet  that  included  rate, 
educational  level,  Navy  schools  attended,  and  shipboard  experience.  Standard 
explanations  and  instructions  were  read  to  the  subjects.  Since  the 
experimenter's  task  was  extremely  tedious,  all  materials  were  organized  for 
easy  administration. 

Dependent  Variable 

The  response  variable  was  the  time  in  seconds  (recorded  to  the  nearest 
tenth)  required  to  complete  each  information  search.  After  reading  the  item 
description  sheet,  the  experimenter  said,  "Go,"  whereupon  the  subject  turned 
to  the  next  drawing  and  the  timing  began.  Timing  continued  until  the  subject 
either  pointed  at  a  part  in  the  drawing  or  began  to  speak  his  response,  as 
required.  If  the  response  was  not  correct,  then  the  experimenter  said, 
"That's  not  it,"  and  resumed  timing. 

Subjects 

Subjects  were  243  Navy  enlisted  men  in  three  different  rating  groups: 
Sonar  Technician  (ST),  Gunner's  Mate  (GM),  and  Boiler  Technician  (BT).  To 
examine  the  effect  of  different  experience  levels  on  performance,  40  ST 
"strikers"  (seamen  and  below  who  are  to  be  rated  as  STs  upon  completion  of 
sonar  "A"  school)  were  tested.  Table  1  compares  these  four  groups  on  a 
variety  of  characteristics.  The  major  factors  noticed  in  this  comparison  are 

(1)  the  expected  dissimilarity  between  the  experienced  and  inexperienced  STs, 

(2)  the  someirfiat  greater  similarity  between  STs  and  BTs  than  between  STs  and 
GMs,  and  (3)  the  expected  lower  levels  of  education  (on  the  average)  for  both 
GMs  and  BTs  as  compared  to  STs. 


Subjects  were  well-motivated  and  cooperative.  Subjects'  spontaneous 
comments  indicated  that  the  stimulus  variations  were  obvious  enough  that  the 
subjects  could  relate  potential  outcomes  of  the  study  to  their  own  job  tasks. 


Summary  of  Subject  Characteristics 


The  selection  of  an  approach  to  data  analysis  must  take  into  account  the 
following  considerations: 

1.  A  cursory  examination  of  the  data  indicated  that  the  distributions  are 
markedly  skewed,  as  would  be  expected  of  time  data  of  this  sort. 

2.  Means  and  variances  of  the  distributions  appear  to  be  correlated 
substantially. 

3.  Because  the  cells  of  the  design  are  divided  among  four  groups  of 
subjects,  the  observations  may  not  be  statistically  independent. 

4.  It  is  necessary  to  make  numerous  tests  of  significance  on  the  data, 
pooling  observations  from  the  same  subjects  and  from  different  subjects  in 
various  combinations. 

Under  these  circumstances,  the  following  was  regarded  as  the  most 
reasonable  approach: 

1.  Use  appropriate  nonparametric  tests.  There  is  evidence  that  the 
correlations  among  observations  are  negligible;  the  use  of  nonparametric  tests 
appears  justified.  The  Mann-Whitney  U  test  for  the  two-sample  case  and  the 
Kruskal-Wallis  test  for  the  k-sample  case  were  selected  because  they  are  the 
most  powerful  nonparametric  tests  of  their  respective  null  hypotheses. 

2.  Be  conservative  in  selecting  a  significance  level.  When  numerous 
tests  are  performed,  it  is  expected  that  some  of  the  "significant"  differences 
will  actually  be  due  to  chance.  This  is  less  likely  if  the  significance  level 
is  conservative.  Consider  using  p  <  .001  as  the  lowest  acceptable  level. 

3.  Mathematically  transform  the  scores  to  reduce  the  skewness  and  render 
the  data  more  amenable  to  a  standard  analysis  of  variance.  The  reciprocal  of 
time  scores  (which  may  be  interpreted  as  a  measure  of  speed)  is  often  used 
with  response  time  data  for  this  purpose. 


RESULTS  AND  DISCUSSION 


Test  Item  Distributions 


Appendix  C  contains  descriptive  data  on  the  distribution  of  scores  for 
each  of  the  160  test  items.  The  tables  in  this  appendix  show  the  highest  and 
lowest  scores,  the  quartiles,  the  range,  he  interquartile  range,  the  number 
of  subjects,  and  the  group  to  which  the  item  was  administered.  These 
distributions  are  displayed  in  groups  of  four.  For  Tasks  1,  2,  and  3  the 
distributions  for  the  four  number-of-callout  steps  are  shown  together;  thus, 
the  effect  of  increasing  the  number  of  callouts  can  be  seen  directly.  For 
Tasks  4  and  5  only  four  test  items  were  administered;  these  are  shown  together 
(Tables  C-39  and  C-40).  For  each  set  of  four  test  items,  the  result  of  the 
Kruskal-Wallis  k-sample  test  is  shown.  Numerous  other  significance  tests  were 
performed  and  are  cited  in  the  text  as  appropriate. 

In  general,  a  large  proportion  of  the  Kruskal-Wallis  and  Mann-Whitney 
tests  indicated  highly  significant  distribution  differences  among  the  various 
conditions.  Since  most  of  the  significant  differences  were  in  the  expected 
direction  and  were  well  beyond  the  .001  level,  it  appears  reasonable  to  accept 
this  as  evidence  that  the  factors  that  were  varied  did  influence  performance 
(search  time). 

In  addition,  six  analyses  of  variance  were  performed  on  the  reciprocals  of 
the  time  scores  (Tasks  1,  2,  and  3  for  E/V  and  C/S  separately).  The  E/V 
analyses  used  five  factors  (number  of  callouts,  sequence,  circling,  extending, 
and  subject  group),  and  the  C/S  analyses  used  four  (number  of  callouts, 
sequence,  presence  of  nomenclature,  and  subject  group).  Interaction  tables 
were  examined  and  compared  with  the  results  of  the  nonparametric  tests. 

Sources  of  Extraneous  Variability 

At  a  number  of  points  in  the  data  analysis,  unexpected  and  inconsistent 
findings  occurred.  Examination  of  the  stimulus  materials  disclosed  two  major 
causes  for  these  unsystematic  results:  (1)  the  position  of  the  target 
information  in  the  table  (for  Task  2  only)  with  resulting  differential  search 
times,  and  (2)  one  or  more  perceptual  phenomena  (such  as  figure-ground  effects 
or  generalized  scanning  strategies)  that  are  systematic  across  subjects  but 
are  unexplained  so  far. 

Table  Search  Time 

In  Task  2  the  subject  must  find  the  part  name  in  a  table,  note  the  callout 
number  associated  with  the  part,  and  then  locate  that  callout  number  on  the 
drawing.  The  time  recorded  for  each  subject  was  measured  from  the  moment  he 
began  to  search  the  table  to  the  moment  he  pointed  to  the  correct  part  on  the 
drawing.  The  total  time,  therefore,  includes  two  separate  activities: 
looking  up  the  part  name  in  the  table  and  searching  for  the  callout  number  on 
the  drawing.  For  task  realism  these  two  activities  were  not  experimentally 
separated  in  the  administration  of  Task  2.  Further,  the  start  of  the 
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subject's  table  search  and  his  pointing  to  the  correct  part  provided  stable, 
reliable  end  points  for  the  measurement  of  the  time  interval. 

A  separate  study  was  conducted  to  determine  the  average  time  required  to 
locate  items  at  varying  "depths"  in  a  table.  Using  a  62-item  table,  16  part 
names  were  selected  at  depths  varying  from  item  number  2  to  item  number  58. 
Part  names  that  might  be  ambiguous  (e.g.,  "roller  mounting  plate,"  "roller 
mount")  were  avoided.  Subjects  were  told:  "Starting  at  the  top  of  the  table, 

find  the  part  called  the  _  and  tell  me  its  callout  number."  Times  were 

measured  from  the  moment  the  subject  oriented  himself  to  the  table  until  he 
began  to  speak  the  callout  number. 

The  results  of  this  table-search  experiment  are  shown  graphically  in 
Figure  9.  The  darkened  circles  represent  the  median  times  required  to  locate 
target  items  in  the  table.  The  line  through  the  data  points  represents  a 
linear  approximation  to  the  data  points.  Each  Task  2  data  point  was  corrected 
by  subtracting  from  the  observed  time  the  average  table  search  time  estimated 
from  the  linear  function  derived  above.  Unless  otherwise  specified,  all 
comparisons  involving  Task  2  data  that  are  made  below  use  the  corrected  times. 

Perceptual  Processes 

The  second  source  of  uncontrolled  variability  is  not  easily  explained  and 
offers  no  method  of  correction.  These  perceptual  phenomena  were  quite 
systematic  across  subjects  and  in  no  way  would  have  been  predicted.  Several 
examples  are  provided  to  describe  their  effects. 

Figure  D-l  is  the  cross-sectional  view  with  44  number  callouts  in  random 
order.  One  Task  1  item  required  that  subjects  locate  and  point  to  the  part 
with  callout  number  14.  Taken  in  isolation,  callout  number  14  actually  stands 
out  from  the  callouts  surrounding  it;  that  is,  once  located,  it  is 
perceptually  prominent.  The  average  time  required  to  locate  this  item, 
however,  ranged  from  7.75  seconds  for  the  STs  to  19.35  seconds  for  GMs ,  with 
BTs  requiring  13.8  seconds.  For  each  of  the  three  groups,  the  time  was  from 
1  to  13  seconds  greater  than  for  a  corresponding  item  on  a  62-callout  drawing. 
It  is  hypothesized  that  the  reason  for  these  unanticipated  findings  is  that  a 
subject  searching  the  drawing  of  Figure  D-l  tends  to  "leap"  from  the 
neighboring  callout  23  to  callout  3  (or  vice  versa)  and  tends  not  to  see 
number  14. 

This  phenomenon  occurs  in  a  similar  fashion  in  certain  tasks  with  the 
exploded  view.  In  Figure  D-2a,  for  Task  1,  the  target  item  is  callout 
number  28.  As  with  the  preceding  example,  the  callout  appears  in  what  might  be 
considered  a  prominent  position.  Compare  this  drawing  with  Figure  D-2b,  the 
same  drawing  but  with  62  callouts.  It  was  hypothesized  from  the  outset  that, 
all  other  things  being  equal,  it  would  take  longer  to  locate  a  target  item 
among  62  callouts  than  among  44  callouts.  The  target  item  on  Figure  D-2b  is 
number  53,  which  appears  to  be  embedded  among  other  numbers  and  should  perhaps 
be  difficult  to  locate.  The  actual  findings  were  the  reverse  of  this 
expectation.  For  each  of  the  three  subject  groups,  it  took  longer  to  locate 
callout  number  28  on  the  44-callout  drawing  than  it  did  to  locate  number  53  on 
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Figure  9.  Plot  of  required  search  time  for  Items  varying  in  location  In  a  table 


t ho  62-callout  drawing.  The  median  times  for  the  62-callout  condition  were 
3.50,  7.10,  and  8.90  lor  groups  ST,  CM,  and  BT,  respectively.  For  44 
callouts,  the  corresponding  times  were  8.50,  15.45,  and  11.00.  The 
explanation  for  the  apparent  difficulty  in  locating  item  number  28  might  be 
similar  to  that  proposed  for  the  cross-sectional  view.  It  may  be  hypothesized 
that,  when  scanning  the  callouts,  a  majority  of  people  begin  with  a 
le f t-to-right  sequence  corresponding  to  normal  reading  habit.  In  Figure  D-2a 
this  abruptly  becomes  a  top-to-bottom  sequence  at  the  upper  right  hand  corner 
of  the  drawing  (the  location  of  callout  number  28).  In  making  this  change  of 
direction,  it  is  hypothesized  that  subjects  tend  to  "cut  the  corner"  and 
overlook  the  target  item.  On  the  62-callout  drawing,  the  number  53  is 
directly  in  the  line  of  scan  (right-to-left  in  this  case)  and  is  therefore 
located  more  easily.  The  callout  arrangement  of  Figure  2  might  alleviate  such 
a  problem,  and  provides  one  type  of  stimulus  pattern  for  checking  the 
hypothesis . 

A  final  example  of  this  problem's  pervasiveness  is  illustrated  by 
Figure  D-3 ,  the  cross-sectional  view  with  both  numbers  and  nomenclature  for 
callouts.  In  Task  2,  the  subject  is  required  to  search  the  callouts  to  locate 
a  part  with  a  specified  name.  In  Figure  D-3,  the  target  item  is  the  FAN.  The 
median  times  required  to  locate  the  FAN  were  11.75,  29.90,  and  36.65  seconds 
for  STs,  BTs ,  and  GMs ,  respectively.  Again,  it  may  be  supposed  that  subjects 
fail  to  perceive  objects  outside  of  a  relatively  narrow  scan  path — a  scan  path 
that  is  determined  by  the  physical  arrangement  of  the  objects  being  scanned. 

In  this  case,  it  would  appear  that  the  scan  path  is  from  item  46  to  58  to  16 
to  48  (or  the  reverse  of  this  order),  and  that  number  30  (the  FAN)  is  masked 
by  this  pattern. 

If  this  explanation  is  valid,  then  there  is  no  easy  "correction  factor" 
that  can  be  applied  to  findings  resulting  from  this  phenomenon  as  there  was 
with  table-search  times.  Further,  from  existing  data  it  is  difficult  to 
predetermine  which  stimuli  might  be  affected  by  this  phenomenon. 

Cross-sectional  View 

The  results  for  the  cross-sectional  view  are  presented  by  subject  task. 

The  major  findings  are  (1)  that,  in  certain  circumstances,  sequential  order  of 
number  callouts  is  extremely  important,  and  (2)  that  user  characteristics 
reflected  in  Navy  ratings  influence  search  times. 


Task  1  Findings 

In  Task  1,  the  instruction  to  the  subject  was  to  point  to  the  part  with  a 
given  callout  number.  For  the  cross-sectional  view,  this  pertains  to  the 
conditions  in  which  the  callouts  are  either  numbers  only  or  both  numbers  and 
nomenclature  (with  a  random  and  sequential  order  in  each  case).  Figure  10 
illustrates  the  findings  in  the  condition  where  callouts  are  numbers  only. 
Since  the  GM  and  BT  samples  had  similar  individual  profiles  for  this 
situation,  they  were  pooled  for  the  purposes  of  this  comparison.  In  the 
condition  with  only  10  callouts,  there  is  virtually  no  difference  in  the  time 
required  to  locate  the  target  information  dependent  upon  either  the  subject 
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Figure  10 


A  comparison  of  Task  1  performance  with  sequence  and 
random  conditions  for  ST  and  GM/BT  groups  combined. 


(rating)  group  or  callout  order.  This  result  is  found  in  nearly  every 
comparison  made,  and  unless  a  particular  point  is  to  be  made  with  regard  to 
the  10-callout  case,  it  will  not  be  discussed  further.  The  10-callout  case 
was  intended  to  represent  a  lower  boundary  condition — a  number  of  callouts 
small  enough  so  that  other  conditions  would  have  little  effect — and  seems  to 
have  functioned  adequately  for  this  purpose. 

Figure  10  illustrates  the  overall  range  of  times  for  the  two  groups  and 
the  two  orders,  rather  than  the  individual  data  points.  With  27  callouts,  a 
divergence  begins  to  be  seen  both  between  sequential  and  random  orders  and 
between  the  different  rating  groups.  Although  the  differences  here  are  not 
statistically  significant  and  probably  are  of  little  or  no  practical 
significance,  there  is  a  tendency  (even  with  as  few  as  27  callouts)  for  more 
time  to  be  required  when  callouts  are  in  random  order  than  when  they  are  in 
sequence.  The  evidence  indicates  that  the  break  point  between  acceptable  and 
unacceptable  practices  (e.g.,  sequential  versus  random  order)  occurs  at  about 
27  callouts. 

When  searching  for  information  among  44  callouts  (and  probably  fewer,  down 
to  some  point  around  27),  the  ordering  of  callouts  in  sequence  becomes  more 
critical.  In  the  case  of  STs,  it  takes  an  average  of  three  to  four  times  as 
long  to  find  a  target  item  when  the  callouts  are  in  random  order  as  when  they 
are  in  sequence.  This  difference  is  of  high  statistical  significance  and  has 
great  practical  significance  as  well.  It  is  also  important  to  note  that 
search  time  itself  is  not  the  only  factor  involved  here.  Users  might  very 
well  become  frustrated  when  unable  to  find  needed  information  quickly,  and 
either  ignore  the  drawing  completely  or  use  it  inefficiently.  In  the 
44-callout  condition  the  STs  and  the  GM/BT  group  differ  greatly  in  the  time 
required  to  locate  the  target  information.  As  was  pointed  out  earlier,  the 
particular  target  item  in  this  case  was  unexpectedly  difficult  to  locate. 
Although  the  target  item  was  identical  for  all  subjects,  the  average  times 
required  to  locate  the  item  were  7.75,  13.80,  and  19.35  seconds  for  the  ST, 

BT,  and  GM  groups,  respectively.  The  difference  between  search  times  for  the 
GMs  and  the  STs  is  statistically  significant,  and  the  difference  for  the  BTs 
and  the  STs  approaches  significance.  These  differences  can  be  explained  only 
in  terms  of  differences  among  subject  groups. 

With  62  callouts,  the  BT  sample  tended  to  take  longer  to  locate  the  target 
information  than  did  the  GMs  or  STs.  The  search  time  differences  between  the 
combined  GM/BT  group  and  the  ST  group  are  not  significant. 

When  numbers  and  nomenclature  both  appear  in  the  callout,  and  when  the 
subject  is  asked  to  find  the  part  with  callout  number  X,  the  GM  and  BT  groups 
tend  to  take  longer  than  the  ST  sample,  but  this  difference  is  probably  too 
small  to  be  of  practical  significance  (e.g.,  an  average  of  about  3  seconds 
with  62  callouts).  Here,  however,  a  general  finding  of  some  importance  is 
that  the  appearance  of  nomenclature  in  the  callout  does  not  seriously 
interfere  with  the  search  for  a  particular  callout  number.  Differences 
between  numbers  only  and  numbers  with  nomenclature  are  generally  small. 
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Task  2  Findings 


Task  2  required  the  subject,  given  the  nomenclature,  to  locate  the  part  on 
the  illustration.  This  involved  searching  the  drawing  directly  when  the 
callouts  were  nomenclature  alone  or  nomenclature  with  numbers,  and  searching 
first  a  table  and  then  the  drawing  when  the  callouts  were  numbers  only.  The 
differences  found  for  Task  2  with  numbers  only  (after  subtracting  table-search 
times)  are  inconsistent.  The  expected  differences  between  sequential  and 
random  orders  occur,  but  little  difference  was  found  among  rating  groups.  One 
exception  to  this  was  the  unexpectedly  long  time  required  for  the  BT  sample  to 
locate  the  target  information  in  the  sequence  condition  with  62  callouts. 

That  task  took  the  BT  group  an  average  of  10.8  seconds,  compared  with  3.95  and 
4.70  seconds  for  GMs  and  STs,  respectively.  With  the  data  available,  this 
difference  cannot  be  explained. 

When  numbers  and  nomenclature  were  both  included  in  the  callout,  the  BT 
sample  required  a  longer  time,  on  the  average,  in  both  the  sequential  and 
random  conditions,  and  the  GM  group  required  a  much  longer  time  in  the  random 
condition  only.  Figure  11  illustrates  the  profiles  for  the  random  condition. 
Note  that,  with  62  callouts,  the  ST  group  required  an  average  of  11.75  seconds 
to  locate  the  target  information;  the  BT  group,  29.9  seconds,  and  the  GM 
group,  36.6  seconds.  Assuming  that  these  results  generalize  sufficiently, 
this  is  a  situation  in  which  consideration  of  user  characteristics  must  have 
top  priority.  Even  the  ST  sample  had  relatively  long  response  times,  and  when 
another  group  takes  three  times  as  long,  it  is  evident  that  relief  is 
required.  At  least  one  of  two  steps  should  be  taken:  (1)  ensure  that 
callouts  are  ordered  in  sequence,  and  (2)  reduce  the  number  of  callouts  on  the 
drawing.  Again,  note  that,  with  as  few  as  10  callouts  (and  perhaps  even  with 
as  many  as  27),  there  is  no  great  problem  for  any  of  the  user  groups  sampled 
even  if  the  callouts  are  in  random  order. 

Numbers  with  nomenclature  in  the  callout  actually  interfered  with  the  task 
when  the  task  required  locating  a  part  by  nomenclature.  This  is  in  contrast 
to  the  reciprocal:  Nomenclature  did  not  interfere  with  locating  a  part  by 
number.  Figure  12  illustrates  this  situation,  giving  the  ranges  of  median 
response  times  across  the  three  main  subject  samples.  The  comparison  for  10 
callouts  shows  the  two  conditions  to  have  a  quite  short  and  almost  entirely 
overlapping  range.  In  the  27-callout  situation,  the  picture  is  unclear,  with 
the  ranges  appearing  unexpectedly  large  relative  to  the  10-  and  44-callout 
ranges.  A  clear  pattern  emerges  for  44  and  62  callouts,  however.  The  ranges 
for  the  nomenclature-only  condition  are  relatively  short  and  the  times 
involved  are  relatively  low.  The  ranges  for  the  nomenclature-plus-number 
conditions  are  nonoverlapping  with  nomenclature  only  and  are  considerably 
higher.  This  suggests  that,  if  the  intended  use  of  the  drawing  is  the 
location  of  information  oh  the  basis  of  part  nomenclature,  then  numbers  should 
be  omitted.  Probably  the' best  arrangement  for  a  Task  2  search,  although  not 
directly  tested  in  this  experiment,  is  to  use  nomenclature  only  if  there  are 
10  callouts  or  fewer,  and  otherwise  to  use  numbers  in  sequence  keyed  to  an 
alphabetical  table. 
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Figure  12.  Ranges  of  median  response  times  given  nomenclature  only  or 
nomenclature  plus  numbers  In  Task  2  for  ST,  BT,  and  GM 
groups  combined. 
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Task  3  Findings 


In  Task  3  the  subject  was  to  tell  the  name  of  a  part  marked  in  red.  As  in 
Task  2,  tables  were  required  for  some  conditions.  For  these,  the  subject 
found  the  callout  number  in  the  drawing  and  used  that  number  to  enter  the 
table.  Since  the  tables  were  organized  according  to  callout  number  sequence, 
the  tables  were  efficiently  designed  for  this  type  of  response,  and  the  times 
are  uniformly  short.  With  10  callouts  there  was  no  difference  between  having 
the  nomenclature  in  the  callouts  and  using  a  separate  cable.  For  larger 
numbers  of  callouts,  there  appears  to  be  an  advantage  to  having  the 
nomenclature  in  the  callouts,  even  when  the  number  of  callouts  is  large.  In 
general,  there  appears  to  be  some  increase  in  search  time  as  the  number  of 
callouts  increases  from  10  to  27,  but  not  much  difference  for  subsequent 
increases . 

For  the  conditions  in  which  the  nomenclature  was  contained  in  the 
callouts,  there  were  no  differences  clearly  attributable  to  an  increased 
number  of  callouts,  and  no  differences  were  expected.  Similarly,  in  the 
number-plus-nomenclature  conditions,  there  was  no  difference  in  identifying 
the  nomenclature  of  the  marked  part  when  the  callouts  were  in  sequential  or 
random  order.  This  finding  does  not  argue  for  the  use  of  random  order  of 
numbers  in  callouts.  As  discussed  under  Task  2  above,  if  the  drawing's  only 
use  is  to  determine  the  nomenclature  of  a  part  whose  location  is  known,  then 
the  use  of  numbers  in  the  callout  is  unnecessary.  If  the  numbers  might  have 
to  be  used  in  addition  to  the  nomenclature,  then  the  results  from  Task  1  and 
Task  2  responses  indicate  that  the  callout  numbers  should  always  be  in 
sequence. 

Summary  of  Findings  for  Cross-sectional  View 

1.  Throughout  the  analysis  of  the  cross-sectional  drawing  results,  GMs 
and  BTs  tended  to  take  longer  to  find  the  target  information  than  did  the  STs. 
Whether  this  is  a  function  of  aptitude  differences  or  of  a  mechanical  versus 
electronic  orientation  is  not  known,  but  it  clearly  points  to  the  fact  that 
optimally  designed  drawings  for  different  user  groups  may  not  be  the  same. 

2.  When  the  number  of  numerical  callouts  reaches  about  27,  their  order 
begins  to  become  important.  Search  time  may  increase  by  a  factor  of  three  or 
four  if  the  callouts  are  not  in  sequence.  It  is  recommended  that,  even  with 
very  few  callouts,  the  numbers  be  organized  in  sequence  so  that  such 
organization  becomes  standard  and  can  be  expected  by  the  user. 

3.  If  the  intended  purpose  of  a  drawing  is  to  locate  a  part  whose  callout 
number  is  known,  then  numbers  alone  or  both  numbers  and  nomenclature  are 
equally  efficient.  If  this  is  the  drawing's  sole  purpose,  the  inclusion  of 
nomenclature  in  addition  to  numbers  would  increase  the  drawing's  cost  without 
significantly  improving  its  utility. 

4.  If  the  intended  purpose  of  a  drawing  is  to  locate  a  part  whose 
nomenclature  is  known,  then  the  inclusion  of  numbers  in  addition  to 
nomenclature  will  interfere  with  the  task. 
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5.  If  the  intended  purpose  of  a  drawing  is  to  determine  the  nomenclature 
of  a  part  whose  location  is  known  (Task  3),  then  there  is  little  to  be  said 
regarding  any  of  the  variables  studied  here,  except  for  the  order  of  items  in 
the  table.  If  numbers  alone  are  used  in  the  callouts,  then  they  should  be  in 
sequence  (for  standardization  purposes)  and  the  items  in  the  table  should  be 
ordered  according  to  callout  numbers  rather  than  alphabetically. 

Exploded  View 

The  results  for  random  versus  sequential  order  of  callouts  were  quite 
conclusive,  as  were  those  for  the  cross-sectional  view.  Circling,  extending 
leaders,  and  using  zones  produced  mixed  and  inconclusive  findings. 

Differences  between  experienced  STs  and  inexperienced  ST  "strikers"  were  not 
significant,  but  differences  were  discovered  between  experienced  STs  and  the 
GM  and  BT  samples. 

Random  (RAN)  and  Sequential  (SEQ)  Order 

The  variations  imposed  upon  the  exploded  view  had  the  form  of  a  2  x  2  x  2 
factorial  design.  The  factors  were  random  versus  sequential  order,  circled 
versus  noncircled  numbers,  and  extended  versus  nonextended  leaders.  As 
expected,  the  order  of  the  number  callouts  was  again  the  most  powerful 
variable  manipulated  in  this  study. 

In  virtually  every  case  in  which  the  number  of  callouts  was  27  or  greater, 
the  time  required  to  locate  a  number  when  callouts  were  in  random  order  was 
significantly  longer  than  when  callouts  were  in  sequence.  For  Tasks  1,  2,  and 
3,  within  each  number-of-callouts  step  and  for  each  subject  group,  the  SEQ 
conditions  were  combined  and  contrasted  with  the  RAN  conditions.  Tables  2,  3, 
and  4  show  median  response  times  according  to  number  of  callouts  and  rating 
group  for  Tasks  1,  2,  and  3,  respectively. 

Task  1  Findings.  For  Task  1  ("Point  to  the  part  with  callout  number  X"), 
Table  2  shows  that  when  the  numbers  are  in  sequence,  the  time  required  to 
locate  a  target  item  is  essentially  no  different  when  there  are  10,  27,  44,  or 
62  callouts.  Figures  13  and  14  illustrate  the  Task  1  performance  of  GMs  and 
BTs  as  compared  to  STs.1  The  curves  for  callouts  in  sequence  are  nearly 
horizontal;  the  differences  that  are  seen  are  nonsystematic  and  statistically 
nonsignificant.  The  fact  that  no  differences  were  found  for  callouts  in 
sequence,  even  between  10  and  62  callouts,  is  itself  an  important  finding. 
Although  one  might  have  expected  a  gradual  increase  in  the  time  required  to 
locate  information  as  the  number  of  callouts  increased,  none  occurred.  The 
small  differences  in  the  profiles  are  most  likely  due  to  experimental  error 
rather  than  to  any  differences  inherent  in  the  stimuli.  Knowing  that  a  coming 


Experienced  STs  will  again  be  used  as  the  anchor  group  in  these 
comparisons.  Inexperienced  ST  strikers  were  sufficiently  similar  to  the 
experienced  group  that  they  will  not  be  discussed  further. 
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Table  2 


Task  1,  Exploded  View,  SEQ  and  RAN 
Median  Response  Times:  ST(Hi)  ,  ST(Lo) ,  GM,  BT 


Rating  Group 

10 

Number  of 

27 

Callouts 

44 

62 

Task 

1  SEQ 

ST  (Hi) 

2.00 

2.50 

2.15 

2.10 

ST  (Lo) 

2.38 

2.63 

2.23 

2.03 

GM 

2.43 

3.55 

2.85 

3.38 

BT 

3.00 

4.60 

2.80 

2.65 

Task 

1  RAN 

ST  (Hi) 

2.55 

3.55 

6.80 

7.65 

ST  (Lo) 

2.85 

4.08 

6.48 

10.83 

GM 

3.17 

4.40 

6.73 

11.10 

BT 

3.15 

5.10 

10.85 

10.95 
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10  27  44  62 

NUMBER  OF  CALLOUTS 

Figure  13.  A  comparison  of  Task  1  performance  of  ST  and  GM  groups  on 
exploded  view,  sequence  and  random  conditions. 
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MEDIAN  RESPONSE  TIME  (SECONDS) 


16 


15 

14 


13 

12 

11 

1G 

9 


7 


6 


5 


4 
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10  27  44  62 

NUMBER  OF  CALLOUTS 

Figure  14.  A  comparison  of  Task  1  performance  of  ST  and  BT  groups  on 
\  exploded  view,  sequence  and  random  conditions. 
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stimulus  would  have  callouts  in  sequence  apparently  alerted  the  subject  as  to 
where  his  search  should  begin  and  how  it  should  proceed.  If  this  orientation 
response  generalizes  to  actual  job  tasks,  then  the  user's  task  would  be 
greatly  simplified  by  the  knowledge  that  callouts  would  always  be  in  sequence. 

When  callouts  are  in  random  order,  on  the  other  hand,  there  is  a 
significant  increase  in  the  time  required  to  locate  information  as  the  number 
of  callouts  increases.  Note  in  Figures  13  and  14  that,  for  all  groups,  the 
time  required  in  the  62-callout  situation  is  three  or  more  times  that  required 
when  there  are  10  callouts.  Similarly,  the  time  required  for  the  62-callout 
condition  when  the  numbers  are  in  random  order  is  from  two  to  four  times  as 
long  as  the  analogous  situation  when  the  numbers  are  in  sequence.  Finally, 
both  the  GMs  and  BTs  tend  to  take  significantly  longer  than  the  STs  when  there 
are  62  callouts,  and  the  BTs  take  significantly  longer  when  there  are  44 
callouts  as  well.  It  is  apparent  again  that  the  GMs  and  BTs  experience  more 
difficulty  than  STs  when  the  number  of  callouts  reaches  44,  and  considerably 
more  difficulty  for  62  callouts. 

f. 

Task  2  Findings.  Task  2  proxies  comparing  GMs  and  BTs  with  STs  can  be 
seen  in  Table  3  and  in  Figures  15  and  16,  respectively.  In  Task  2  the  subject 
must  search  for  a  part  name  in  a  table,  determine  the  callout  number  for  that 
part,  and  then  locate  that  number  on  the  drawing.  As  mentioned  earlier,  the 
time  increment  (based  on  a  study  of  STs)  required  to  search  the  table  is 
subtracted  out  of  total  time  before  comparisons  are  made  on  this  task. 

Figures  15  and  16  show  only  this  corrected  time.  When  callouts  are  in 
sequence,  there  is  a  much  greater  tendency  toward  an  increase  in  the  time 
required  as  the  number  of  callouts  increases.  The  time  required  (after  the 
correction  has  been  applied)  is  up  to  twice  as  long  for  62  callouts  as  for  10 
callouts.  Since  the  corrected  times  for  Task  2  should  follow  the  same  pattern 
as  the  actual  times  for  Task  1  (i.e.,  no  increase  in  time  as  the  number  of 
callouts  increases),  this  systematic  increase  requires  some  explanation.  The 
simplest  hypothesis  is  that  the  table-search-time  correction  applied  to  these 
data  is  to  some  degree  inappropriate.  The  linear  function  representing  the 
correction  was  derived  from  data  on  STs.  The  function  may  in  fact  be 
nonlinear,  and  a  different  function  may  be  required  for  the  other  rating 
groups.  More  investigation  is  needed. 

The  differences  between  SEQ  and  RAN  for  27,  44,  and  62  callouts  are 
significant  and  sizable  for  each  rating  group.  Furthermore,  the  differences 
between  both  GM  and  BT  samples  and  the  ST  sample  with  62  callouts  amount  to  an 
average  of  9  seconds  for  GMs  and  almost  5  seconds  for  BTs.  The  differences 
with  44  callouts  are  less  pronounced  for  the  GMs  but  are  even  more  extreme  for 
the  BTs.  This  finding — that  the  mechanical  ratings  in  general  require  more 
time  to  locate  information  when  as  many  as  62  callouts  are  present — is 
persistent,  and  may  have  serious  implications  for  the  construction  of  drawings 
for  use  by  these  ratings. 

Task  3  Findings.  As  in  the  Task  3  analysis  of  the  cross-sectional  view, 
there  is  no  consistent  difference  between  sequential  and  random  orders  (see 
Table  4),  nor  would  one  be  expected.  There  is  a  slight  effect  that  depends 
upon  the  number  of  callouts  on  the  drawing.  There  is  a  tendency  among  all 
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Table  3 

Task  2,  Exploded  View,  SEQ  and  RAN 
Median  Response  Times:  ST(Hi),  ST(Lo),  GM  BT 


Number  of 

Callouts 

Rating  Group 

10 

27 

44 

62 

Task  2  SEQ 

ST (Hi) 

1.38 

2.10 

2.55 

3.05 

ST(Lo) 

1.20 

2.30 

5.25 

3.40 

GM 

2.27 

2.38 

2.75 

4.85 

BT 

2.23 

3.18 

5.73 

4.63 

Task  2  RAN 


MEDIAN  RESPONSE  TIME  (SECONDS) 


Figure  16.  A  comparison  of  Task  2  performance  of  ST  and  BT  groups  on 
exploded  view,  sequence  and  random  conditions. 
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Table  4 


Task  3,  Exploded  View,  SEQ  and  RAN 
Median  Response  Times:  ST(Hi),  ST(Lo),  GM,  BT 


Number  of 

Callouts 

Rating  Group 

10 

27 

44 

62 

Task  3  SEQ 

ST  (Hi) 

2.80 

3.20 

3.20 

4.00 

ST  (Lo) 

3.38 

3.53 

3.50 

4.08 

GM 

3.15 

4.40 

4.20 

4.60 

BT 

3.50 

3.55 

4.10 

5.00 

Task 

3  RAN 

ST  (Hi) 

2.40 

3.05 

3.35 

3.35 

ST  (Lo) 

2.83 

3.40 

3.65 

4.13 

GM 

2.90 

3.73 

4.58 

4.78 

BT 

3.25 

4.00 

4.15 

4.50 

47 


subjects  to  take  longer  to  determine  the  nomenclature  of  a  part  marked  on  the 
drawing  as  the  number  of  callouts  increases.  It  is  hypothesized  that  this 
difference  (which  may  be  too  small  to  be  of  concern)  is  due  to  two  factors. 
First,  as  the  number  of  callouts  increases,  there  is  a  greater  likelihood  of 
misreading  the  number  of  the  target  item,  due  simply  to  clutter.  Subjects 
were  observed  taking  considerably  more  care  with  large  numbers  of  callouts  to 
ensure  that  they  did  not  make  mistakes.  Second,  there  is  certain  to  be  some 
increase  in  the  time  required  to  locate  a  callout  number  deep  in  a  table  as 
opposed  to  near  the  top  of  the  table.  This  is  not  as  critical  a  function  as 
searching  for  the  name  of  a  part  in  a  table,  for  which  the  table-search 
correction  was  applied  in  Task  2.  It  merely  involves  locating  a  particular 
number  in  a  column  of  numbers  arranged  in  numerical  order  and  reading  the  part 
name  associated  with  that  number.  The  time  involved  is  probably  quite  small 
and  is  perhaps  not  of  practical  significance.  At  any  rate,  it  is  difficult  to 
see  how  tables  could  be  organized  differently  to  make  the  process  easier. 

Circling  Callouts  and  Extending  Leaders 

Circling  number  callouts  and  extending  leaders  to  enhance  visual  scanning 
were  expected  to  have  much  weaker  effects  than  the  sequence-random  variable. 

In  fact,  it  was  anticipated  that,  if  the  numbers  were  in  sequence,  there  would 
be  no  effect  due  to  these  "cosmetic"  variations.  If  this  were  the  case,  then 
the  C  and  £  variables  would  prove  beneficial  only  when  the  numbers  were  in 
random  order  and  when  there  were  many  callouts  to  scan.  To  test  this 
hypothesis,  both  the  sequence  and  random  conditions  for  44  and  62  callouts. 
Tasks  1  and  2,  were  examined.  The  results  were  mixed  and  generally 
inconsistent . 

Figure  17  shows  the  profiles  of  median  response  times  for  Task  1  with  44 
callouts  in  random  order.  This  is  probably  the  only  instance  from  which 
clearcut  conclusions  might  have  been  drawn.  With  the  exception  of  the  dip  in 
the  profile  for  GMs  for  the  circled/not  extended  (C/NE)  condition,  these 
results  are  what  would  have  been  predicted.  This  figure  shows  the  extension 
variable  to  be  the  more  powerful  of  the  two,  and  circling  to  be  superior  to 
not  circling.  In  the  extended  condition,  callouts  are  placed,  for  the  most 
part,  in  straight  vertical  and  horizontal  lines  near  the  outer  edges  of  the 
illustration  (e.g.,  Figure  D-2).  Under  these  circumstances,  one  would  presume 
that  the  search  for  a  particular  callout  number,  without  the  cues  provided  by 
having  the  callouts  in  sequence,  would  be  enhanced  by  the  ability  to  scan  in 
straight  lines  across  the  top  and  bottom  and  up  and  down  the  sides  of  the 
drawing.  The  results  in  this  particular  instance  (Figure  17)  would  seem  to 
bear  out  that  hypothesis:  The  best  performance  is  attained  when  the  leaders 
are  extended  and  the  numbers  are  circled,  and  the  worst  performance  is 
attained  when  leaders  are  not  extended  and  the  numbers  are  not  circled.  The 
worst  performance  for  each  group  (NC/NE)  is  two  to  three  times  as  long  as  the 
best  performance. 

Figure  18  profiles  the  median  response  times  for  Task  1  with  62  callouts 
in  sequence,  and  shows  much  of  the  inconsistency  found  in  these  results. 
Results  for  the  ST  and  BT  samples  indicate  that,  even  with  the  callouts  in 
sequence,  some  effect  of  the  C/E  variation  is  present.  The  conclusion  drawn 
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MEDIAN  RESPONSE  TIME  (SECONDS) 


1 


CALLOUT  PROPERTY  VARIATION 

Figure  17.  A  comparison  of  ST,  GM,  and  BT  performance  in  the  four  circle- 
extend  conditions.  Task  1,  44  callouts  in  random  order. 


49 


I _  «  _ I  I  . 

CIRCLED/  NOT  CIRCLED/  CIRCLED/  NOT  CIRCLED/ 

EXTENDED  EXTENDED  NOT  EXTENDED  NOT  EXTENDED 

CALLOUT  PROPERTY  VARIATION 


18.  A  comparison  of  ST,  GM,  and  BT  performance  in  the  four  circle- 
extend  conditions.  Task  1,  62  callouts  in  sequence. 


from  these  profiles  is  that  there  is  a  negative  effect  on  scanning  efficiency 
if  the  callouts  are  neither  circled  nor  extended.  Again,  the  uncircled, 
embedded  callout  numbers  in  this  condition  would  be  expected  to  be  masked  by 
the  features  of  the  drawing  itself.  According  to  this  figure,  there  is  no 
detrimental  effect  caused  by  simply  not  extending  the  leaders;  performance  is 
degraded  only  when  the  nonextended  leaders  are  also  not  circled.  None  of  the 
effects  indicated  here  for  the  sequence  condition  are  as  strong  as  those  found 
for  random  ordering,  as  expected.  The  profile  for  the  GM  sample  leads  to 
entirely  different  conclusions  than  those  of  the  STs  and  BTs.  The  differences 
noted  cannot  be  reconciled  using  the  available  data. 

A  detailed  examination  of  all  of  the  results  involving  circling  and 
extending  might,  if  nothing  else,  lead  to  questions  that  should  be 
investigated  more  completely.  Figure  19  points  to  one  possible  area  of 
interest.  The  profile  for  STs  generally  conforms  to  that  in  Figure  19,  with 
62  callouts  in  sequence;  that  is,  it  indicates  little  (if  any)  difference 
among  the  variations  except  for  the  tendency  for  the  NC/NE  stimulus  to  require 
a  longer  search  time.  Perhaps  the  most  intriguing  feature  of  Figure  19, 
however,  is  the  similarily  of  the  GM  and  BT  profiles  and  the  distinct 
difference  between  both  of  these  and  the  STs.  The  circled/nonextended 
condition  in  this  case  involved  a  rather  difficult  task:  locating  callout 
number  11,  which  is  deeply  embedded  in  the  outlines  of  the  drawing  itself. 
Interestingly,  STs  appeared  to  have  very  little  difficulty  in  locating  it, 
whereas  both  the  GMs  and  BTs  required  about  four  times  as  long.  These 
differences  are  highly  significant,  and  raise  the  question  of  possible 
differences  in  abilities  of  the  mechanical  ratings  compared  to  the  electronics 
rating.  Much  more  investigation  is  required  to  determine  the  nature  of  these 
differences  and  their  implications  for  graphic  design  for  the  various  ratings. 

Although  conclusions  from  the  circle/extend  conditions  cannot  be  drawn 
with  confidence,  the  overall  data  indicate  that  circling  callout  numbers  and 
extending  leaders  are  weak  variables  at  best,  even  when  the  number  of  callouts 
is  large,  and  especially  if  the  callout  numbers  are  in  sequence.  The  results 
do,  however,  again  suggest  that  user  characteristics  reflected  in  Navy  rating 
should  influence  the  selection  of  graphic  presentation  features. 

Zones 


The  tasks  performed  by  the  subjects  in  this  study  did  not  lend  themselves 
to  the  use  of  zones.  The  subjects  frequently  seemed  confused  on  these  test 
items  and  the  results  are  not  enlightening.  Comparing  the  Task  4  and  5  items 
with  their  corresponding  Task  1  and  2  items,  the  items  with  zones  invariably 
had  longer  search  times.  Zones  apparently  are  not  useful  for  locating  parts 
when  a  number  callout  must  also  be  used  for  verification. 

There  were  no  significant  differences  among  the  Task  4  distributions.  In 
Task  5,  the  C/E  cell  was  significantly  lower  than  the  others,  which  would  be 
reasonable  except  that  the  overall  data  on  circling  and  extending  leaders 
makes  this  finding  suspect. 
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CALLOUT  PROPERTY  VARIATION 

Figure  19.  A  comparison  of  ST,  GM,  and  BT  performance  in  the  four  circle- 
extend  conditions.  Task  1,  44  callouts  in  sequence. 
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Cross-sectional  versus  Exploded  Views 


The  NUM-SEQ  and  NUM-RAN  conditions  of  the  cross-sectional  drawing  were 
similar  to  the  exploded-view  drawings  in  that  they  had  no  nomenclature.  This 
allowed  a  comparison  between  responses  to  the  two  drawings.  For  each 
number-of-cal louts  step,  a  comparison  was  made  between  NUM-SEQ  and  the 
combined  SEQ  conditions  for  the  exploded  view,  and  between  NUM-RAN  and  the 
combined  RAN  conditions  for  the  exploded  view.  Of  the  24  comparisons,  11  were 
not  significant  at  the  .05  level  and  5  were  significant  at  the  .001  level.  Of 
these  five,  four  indicated  somewhat  longer  times  for  the  cross-sectional  view. 
The  conclusion  is  that  responses  to  the  two  drawings  were,  for  the  most  part, 
very  similar;  either  type  of  drawing  could  be  used  in  this  type  of  study. 
Confusing  elements  in  this  particular  cross-sectional  drawing  probably 
accounted  for  the  significant  differences. 

Independence  of  Observations 

Correlations  might  exist  in  the  data  because  the  same  subjects  were  used 
for  groups  of  test  items.  If  speed  of  response  is  a  subject  trait,  and  one  of 
the  groups  of  subjects  happened  to  have  a  large  proportion  of  fast  subjects, 
this  would  be  reflected  in  conditions  scattered  throughout  the  experiment, 
causing  significant  differences,  not  because  of  stimulus  characteristics,  but 
because  of  spurious  subject  group  characteristics.  Correlated  data  could  thus 
generate  an  excess  of  significant  sample  differences  where  no  differences 
really  exist. 

If  search  speed  is  not  a  subject  trait,  but  is  instead  associated  only 
with  test  items,  then  the  observations  can  be  considered  essentially 
independent  for  the  purpose  of  performing  other  analyses.  To  assess  the 
potential  correlations  in  these  data,  three  analyses  were  performed  on  the  ST 
data  using  the  Kendall  coefficient  of  concordance  method  to  calculate  average 
Spearman  rank  difference  correlations. 

The  first  analysis  may  be  thought  of  as  pairing  the  36  subjects  in  each 
group  to  yield  all  possible  pairs,  calculating  the  rank  difference  correlation 
(rho)  across  the  40  test  items,  and  finding  the  average  rho  of  the  630  pairs. 
The  average  correlations  turned  out  as  follows: 

Group  Average  Rho 

A  .74 

B  .70 

C  .73 

D  .71 

These  average  correlations  are  significantly  different  from  zero  well  beyond 
the  .001  level. 

The  interpretation  of  such  a  substantial  average  correlation  is  that  the 
subjects  reacted  to  the  items  similarly;  i.e.,  items  with  short  times  for  one 
subject  tended  to  have  short  times  for  the  others.  There  were  "hard"  items 
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and  "easy"  items;  response  time  is  definitely  related  to  the  test  item. 
Additionally,  the  close  agreement  between  groups  is  an  indication  that  the 
groups  were  equivalent . 


The  second  analysis  was  similar,  except  that  the  test  items  were  taken 
pairwise,  with  the  correlations  calculated  across  subjects.  In  this  case,  a 
high  average  correlation  would  mean  that  subjects  are  individually  consistent; 
that  is,  there  are  fast  subjects  and  slow  subjects.  Low  correlations  would 
indicate  that  subject  responses  tended  to  be  independent  of  the  subject,  with 
very  little  carryover  from  one  test  item  to  another. 

Group  Average  Rho 

A  .16 

B  .13 

C  .17 

D  .14 

These  correlations  are  also  significant  beyond  the  .001  level,  but  are 
uniformly  low,  indicating  almost  no  relationship  between  test  items. 

Taken  together,  these  results  indicate  that  it  is  justifiable  to  regard 
all  the  scores  as  essentially  Independent  measures  reflecting  stimulus 
characteristics . 

A  third  analysis  confirmed  the  second.  Because  of  the  way  the  items  were 
assigned  to  groups,  certain  similar  items  were  administered  to  the  same 
subjects.  Specifically,  there  were  10  pairs  of  Task  1  items  of  this  type,  in 
which  the  two  items  in  the  pair  differed  only  in  having  either  27  or  62 
callouts.  The  correlation  was  calculated  for  each  of  the  10  items,  yielding 
rhos  ranging  from  -.08  to  .33.  They  are  uniformly  low,  and  the  highest  falls 
to  meet  the  .05  level  of  significance,  further  indicating  the  independence  of 
the  observations. 

Value  of  the  Results 


Three  questions  might  be  raised  about  this  investigation: 

1.  Aren't  the  results  intuitively  obvious? 

2.  Will  the  results  generalize  to  the  work  environment? 

3.  Even  the  longest  search  times  are  quite  short;  will  these  proposed 
guidelines  really  make  much  difference? 

The  major  findings — that  numbers  are  easier  to  find  if  they  are  in 
sequence,  that  tables  should  be  organized  alphabetically  or  numerically 
depending  on  the  user's  entry  information,  and  that  drawings  should  be 
tailored  to  different  uses — seem  rational  and  obvious.  The  importance  of  this 
study  is  twofold:  It  provides  the  beginnings  of  an  objective,  numerical 
measure  of  the  cost  of  violating  these  rational  principles;  and  it 
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demonstrates  that  the  guidance  and  requirements  found  in  current  military 
procurement  documents  often  do  not  conform  to  the  results  of  this  study. 

With  regard  to  the  generalizability  of  the  findings,  it  should  be  noted 
that  the  subjects '  task  in  the  experiment  was  not  truly  a  job  task  simulation, 
but  rather  a  small  part  of  an  actual  job  task.  However,  the  search  times 
obtained  in  the  study  are  probably  underestimates  of  on-the-job  times,  since 
the  subjects  were  highly  motivated  to  "beat  the  clock"  in  spite  of  instruc¬ 
tions  to  relax  and  pace  themselves  normally.  Ordinary  distractions  and 
discomforts  of  the  work  environment  were  absent,  and  subjects'  whole  attention 
was  on  the  task  itself.  It  is  probable  that,  in  the  work  environment,  the 
same  relative  magnitudes  or  ratios  would  appear,  but  the  actual  search  times 
would  be  longer. 

Finally,  even  though  the  time  for  each  information  search  is  short,  these 
are  small  tasks  that  occur  with  great  frequency,  so  the  total  time  could  be 
appreciable.  There  is  also  an  annoyance  factor  that  was  observed  but  not 
measured  or  recorded  during  the  data  collection;  that  is,  subjects  who  had 
trouble  finding  a  number  or  nomenclature  sometimes  became  quite  agitated.  On 
the  job,  if  the  use  of  a  drawing  appears  to  make  the  job  longer  instead  of 
shorter,  or  harder  instead  of  easier,  then  the  technician  may  reject  the 
drawings  or  the  entire  manual  whenever  possible.  Every  effort  should  be  made 
to  make  the  data  in  technical  manuals  as  accessible  and  helpful  to  the  user  as 
possible. 

Evaluation  of  the  Study 

The  purpose  of  this  study  was  not  only  to  generate  data  on  which  to  base 
guidelines,  but  also  to  evaluate  the  feasibility  of  attempting  an  objective 
assessment  of  graphic  comprehension  or  usefulness.  As  a  prototype  for 
future  investigations,  the  study's  approach  appears  to  have  excellent 
possibilities  for  solidifying  comprehension  requirements  in  technical  manual 
procurement  documents.  The  approach  is  different  from  other  work  in  this 
area,  because  it  focuses  directly  on  the  behavior  of  the  technician  as  he 
attempts  to  extract  needed  data  from  a  drawing.  It  does  not  attempt  a 
theoretical  formulation  of  graphic  comprehension,  nor  does  it  attempt  to 
operationally  define  stimulus  features  such  as  "density"  and  "clutter."  It 
asks  what  the  technician  is  doing  with  the  drawing,  and  what  factors  might 
Influence  that  activity.  This  method  has  the  virtue  of  generating  empirical 
relationships  that  should  have  direct  application  and,  it  is  hoped,  will 
provide  a  basis  for  theoretical  developments  regarding  perception  and  human 
Information  processing  in  complex  environments. 

The  most  serious  shortcoming  in  this  study  was  its  failure  to  control 
adequately  for  the  two  sources  of  extraneous  variation:  The  location  of  the 
targets  on  the  drawings  and  the  position  of  the  information  in  the 
accompanying  tables.  The  Impact  of  target  location,  both  on  the  illustration 
and  in  the  table,  was  greatly  underestimated.  It  is  hypothesized  that 
systematic  perceptual  variations  caused  some  target  items  to  take  much  longer 
to  locate  than  others  (notably  in  the  random  conditions)  and  confounded 
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certain  results.  Table  search  time,  of  course,  is  part  of  the  total 
performance  time,  but  based  on  data  dealing  only  with  table  search  time,  this 
effect  can  be  dealt  with  mathematically. 


Overall,  the  practical  impact  of  these  deficiencies  is  limited.  The 
sequence- random  effect  was  very  strong,  and  leads  to  the  conclusion  that 
callouts  should  be  arranged  in  sequence  at  all  times.  The  hypothesized 
perceptual  phenomenon  is  a  problem  only  in  the  random  conditions,  which  the 
results  of  this  study  suggest  should  never  be  used.  It  poses,  at  this  time, 
an  interesting  theoretical  question  but  not  a  practical  one. 


56 


CONCLUSIONS  AND  RECOMMENDATIONS 

1.  For  part  location  by  callout  number,  callouts  should  always  be 
arranged  In  numerical  sequence  on  the  drawing,  even  when  the  number  of 
callouts  is  very  small.  When  callout  numbers  are  given  in  procedural  steps, 
they  should  be  in  sequence  on  the  drawing  rather  than  in  the  text.  If  the 
numbers  are  in  sequence,  then  the  number  of  callouts  may  be  quite  large, 
certainly  greater  than  the  62  callouts  used  in  this  study. 

2.  For  part  location  by  nomenclature,  nomenclature  callouts  may  be  used 
if  the  number  of  callouts  is  10  or  less.  Otherwise,  an  alphabetical  table 
should  be  provided  to  key  the  nomenclatures  to  number  callouts,  which  should 
be  in  sequence  on  the  drawing. 

3.  For  part  identification  (finding  the  nomenclature  when  the  location  is 
known),  nomenclature  callouts  are  superior  to  numbers  keyed  to  a  table  even 
when  the  number  of  callouts  is  so  large  that  the  drawing  appears  excessively 
cluttered. 

A.  If  the  numbers  are  in  sequence,  then  devices  to  enhance  the 
discrimination  and  visual  scanning  of  number  callouts,  such  as  circling  and 
aligning  the  numbers,  are  probably  unnecessary. 

5.  Zones  are  not  useful  for  locating  parts  when  a  number  callout  must 
also  be  used  for  verification. 

6.  Since  the  guidelines  differ  with  the  type  of  information  search, 
drawings  must  be  designed  for  specific  information-search  tasks. 

7.  There  appears  to  be  a  consistent  difference  between  the  combined  GM/BT 
sample  and  the  ST  sample  with  regard  to  overall  information-search  performance. 
Since  this  occurs  also  in  tasks  not  involving  reading,  more  general  perceptual 
or  behavioral  factors  are  responsible.  These  need  to  be  identified  and 
investigated. 

8.  Although  isolating  the  technical  manual  user's  information-search 
behaviors  and  varying  the  features  of  drawings  that  influence  search  speed 
and  accuracy  appear  very  promising,  care  must  be  taken  in  future  studies  to 
randomize,  counterbalance,  or  measure  the  effect  of  target  location  in  the 
stimulus  materials.  Inadequate  control  of  this  variable  in  the  present  study 
caused  confounding  in  certain  desired  comparisons,  but  need  not  cause  problems 
in  future  studies. 

9.  Clarification  of  the  graphic  comprehension  issue  should  be  pursued 
through  empirical  studies  of  users'  information-search  behavior  and  the 
stimulus  variations  that  influence  its  efficiency. 
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APPENDIX  A 


SAMPLES  OF  DRAWINGS  SHOWING  EXPERIMENTAL  VARIATIONS 


POLE 

SHOES  (2) 


COMMUTATOR 


ROTOR 


BRAKE 

LINING 

MP7 


Figure  A-l . 
10  Callouts, 


Cross-sectional  View, 
Nomenclature  Only. 


Parts  Location  Index 


CALLOUT  REF 

number  desig 


NAME 


1 

MP13 

0-Ring 

2 

MP4 

Ball  Bearing 

3 

MP28 

Stem  Bushing 

4 

MP22 

Bearing  Spacer 

5 

MP20 

Handwheel 

6 

MP3  5' 

Spring  Plunger 

7 

MP31 

Magnet 

8 

MP33 

Roll  Pin 

9 

MP23 

End  Bracket 

10 

Socket  Head  Set  Screws 

11 

MP27 

Brake  Magnet  Spacer 

12 

MP19 

Threaded  Rods 

13 

MP30 

Brake  Coil  Magnet 

14 

E4 

Armature 

15 

MP32 

Frame 

16 

J1 

Receptacle  w/Gasket 

17 

MP10 

Pole  Shoes 

18 

MP26 

Movable  Brake  Disc 

19 

E2 

Brush 

20 

MP21 

Brush  Holder  Plates 

21 

MP11 

Shaft 

22 

E6 

Commutator 

23 

LI 

Brake  Coil 

24 

E8 

Rotor 

25 

MP7 

Brake  Lining 

26 

MP1 

Shaft  Seal 

27 

MP17 

Bearing  Bracket 

Figure  A-2.  Cross-sectional  View, 
27  Callouts,  Numbers  Only,  Random. 


APPENDIX  B 


EXAMPLES  OF  POORLY  CONSTRUCTED  ILLUSTRATIONS 
FROM  TECHNICAL  MANUALS 


NAVSHIPS  09(57-320-4040 


>OA /s 

asai  (roll) 


ath  Drive  Unit  (2A2A1A2)  or  Roll  Drive  Unit  (2A2A3A2), 
Disassembly 

Figure  B-l.  Example  of  poor  callout 
arrangement  and  use  of  hand  lettering. 

B- 1/2 


r  .  '>/)  TO 

/»i rL  TO  ts) 

•o  eeojt(.T  „</r 


SM  / 


compressifl 
engineeriH 
callout  a» 


ORIGINAL 


Bja 

* 

iSS5H 

ir^ 

/SHIPS  0967-320-4050 


:,a  7e*s/;rt  ■ 


3  ***** -J  4  (  6'  f*. 

5  '/V  2  <*«'  S-X£X 

i.i  &/*  >£**  *'0*  **■£*  •**rtO 
ss/^' 


Of  f*°l 

J4tC9+**/l  ,C*r 
/*/*  ac'  T  fct 
A  m  ($)  0*1'  -  *  "<WJ 


NAVSEA  OP  3713  VOLUME  7  FIRST  REVISION 


Part  Number 


1  2 


3  4  5  6  7 


Description 


Units 

per 

Assy 


Usable  on 


Mk 


Mod 


Code 


LD  631239 
Dwg  2873065 

1611209-208 


LOWER  HOIST  POWER  DRIVE  UK  77  MOD  0 
(Fig.  278-23  NHA) 

(Attaching  Parts) 

SCREW,  Self-locking  sell  cap,  3  4-10  by  1-3  4m.,.. 


10 


1368177-1 

1611209-90 

650206-101 

650207-1 

1550306-1 

1611209-117 

733543-6 

1320749-17 

734676-835 

MS28775-210 

733543-5 

LO  631348 

LD  631242 

LD  631243 

LD  631241 

LD  631240 

1594000-2 

2815179-31 

8-Z-1131-16 

LD  631346 


PIPE  ASSY,  1  8  IPS,  3.0  ft  7-39  64  m . 

SCREW,  Self-locking  sch  cap.  5  16-18  by  1.0  in.  .. 

ADAPTER,  Square  side  outlet  pipe  . 

GASKET,  Synth  rubber .  . 

RING,  Gasket  retaining  . . 

.  SCREW,  Self-locking  sch  cap,  3  8-16  by  2-3  4  m. 

ADAPTER,  Square  side  outlet  pipe  . . 

PIPE,  Seamless . 

TAG,  Ident  . . 

PACKING,  Preformed . . 

BUSHING,  Filter . 

VALVE  BLOCK,  Accumulator  charging  (See  Fig.  293).  .  . 

FILTER,  Accumulator  servo  (See  Fig.  294) . 

FILTER,  Accumulator  supercharge  (See  Fig.  295) . 

HOUSING,  Accumulator  solenoid  (See  Fig.  296)  . 

VALVE,  Control  (See  Fig.  297) . 

SCREW,  Drive,  No.  0  by  3  16  in . 

PLATE,.  Ident  .  . 

PLATE,  Ordalt . 

TANK,  Accumulator  (See  Fig.  298)  . 


1 

4 

1 

1 

1 

4 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

8 

1 

1 

1 


Figure  B-4.  Example  of  poor  callout 
arrangement  and  inconsistent  use  of 
circling  and  "stacking." 
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4-51  EJECTOR  DUMP  VALVE  REMOVAL  AND  INSTALLATION. 


4-52  REMOVAL.  (see  Figure  4-12) 

a.  Disconnect  actuating  link  from  right 
nosewheel  dor  (NAVAIR  01-45AAE-2-4) . 

b.  Open  access  2222-4  and  2212-6. 

c.  Remote  heat  exchanger  exhaust  duct 
coupling  (1) . 

d.  Remove  nuts  (2),  washers  (3),  shims 
(4),  and  boJts  (5)  securing  duct  to  mounting 
bracket.  Remove  exhaust  duct  (6)  from  air¬ 
plane  . 

e.  Cut  lockwire  and  remove  coupling 
(7),  retainer  rings  (8),  O-rings  (9),  and 
retainer  halves  (10)  securing  turbine  out¬ 
let  duct  (11)  to  water  separator. 

f.  Remove  coupling  (12)  and  0-ring  (13) 


g.  Remove  clamp  (14)  from  oontrol  air 


line. 


h.  Supporting  turbine  outlet  duct,  re¬ 
move  bolt  (15)  and  washer  (16)  attaching  duct 
to  head  exchanger  bracket  and  remove  turbine 
outlet  duct. 

i.  Disconnect  and  remove  drain  line  (17) 
between  control  air  valve  and  head  exchanger 
outlet. 

j.  Disconnect  control  air  lines  (18, 

19,  and  20)  from  ejector  dump  valve. 

k.  Remove  nuts  (21),  bolts  (22),  and 
washers  (23)  from  clamps  (24)  securing  control 
valve  mounting  bracket  to  heat  exchanger 
support  (25) . 

l.  Remove  bolts  (26),  nuts  (27),  and 
washers  (28)  securing  vilve  to  mounting 
bracket . 

m.  Remove  valve  (29)  and  mounting 
bracket  (30) . 

n.  Remove  unions  (31)  and  0-rings 
(32)  from  valve 

o.  Note  position  of  elbow,  loosen  jam- 
nut  (33),  and  remove  elbow  (34)  and  0-ring 
(35)  from  valve. 


Ejector  Dump  Valve  Removal  and  Installation 


IM  MU  M41  *  SJ 


Figure  B-7.  Example  oe  callout  crcwd;rtc 
resulting  in  an  extremely  difficult 
drawing. 
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APPEND  T^.  C 

TEST-ITEM  DISTRIBUTION  STATISTICS 

For  each  table  a  Kruskal-Wallls  k-sample  test  of  distribution  differences 
was  performed,  yielding  H,  which  Is  distributed  as  Chi-square  with  k-1  degrees 
of  freedom.  The  result  Is  Indicated  under  each  table.  Tables  for  Tasks  2  and  5 
show  values  corrected  for  table-search  time. 


Table  C-l 


Test  Item  Distribution  Statistics 
Task  1  (Point  to  Part  Given  Callout  Number) 
CROSS-SECTIONAL  VIEW:  NUM-SEQ 


Sonar  Technician  Sample 


Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

0.30 

0.30 

0.70 

0.90 

Q1 

1.15 

1.00 

1.25 

1.70 

Q2  (Median) 

1.55 

1.30 

1.80 

2.35 

Q3 

2.00 

2.20 

2.60 

3.05 

HI  Score 

3.60 

3.20 

3.60 

6.50 

HI-LO 

3.30 

2.90 

2.90 

5.60 

Q3-Q1 

0.85 

1.20 

1.35 

1.35 

N 

36 

36 

36 

36 

Group 

A 

D 

C 

D 

Note.  H  -  19.98, 

p  <  .001  (p  m 

.001  for  H 

>  16.27) 

Combined 

Boiler  Technician 

and  Gunner 

'  s  Mate  Sample 

Statistic 

Number  of 

Callouts 

10 

27 

44 

62 

LO  Score 

1.00 

1.10 

2.10 

1.30 

Q1 

1.15 

1.30 

2.90 

2.65 

Q2  (Median) 

1.80 

2.90 

3.20 

3.60 

Q3 

2.05 

3.20 

4.25 

6.45 

HI  Score 

3.50 

5.30 

8.20 

15.90 

HI-LO 

2.50 

4.20 

6.10 

14.60 

Q3-Q1 

0.90 

1.90 

1.35 

3.80 

N 

15 

15 

15 

15 

Group 

A 

D 

C 

D 

Note.  H  -  21.22, 

p  <  .001  (p  ■  .001  for  H  > 

16.27) 

C-l 


Table  C-2 


Test  Item  Distribution  Statistics 
Task  1  (Point  to  Part  Given  Callout  Number) 
CROSS-SECTIONAL  VIEW:  NUM-RAN 


Sonar  Technician  Sample 


Statistic 


Number  of  Callouts 


10 

27 

44 

LO  Score 

0.50 

0.90 

1.10 

Q1 

1.25 

3.20 

4.95 

02  (Median) 

2.00 

3.80 

7.75 

Q3 

2.70 

4.80 

15.60 

HI  Score 

14.50 

40.20 

HI-LO 

4.50 

13.60 

39.10 

Q3-Q1 

1.45 

1.60 

10.65 

N 

36 

36 

36 

Group 

B 

A 

D 

Note.  H  -  69.26, 

p  <  .001  (p  * 

.001  for  H 

>  16.27) 

1 

4 
6 
9 

18 

17 

5 


Combined  Boiler  Technician  and  Gunner's  Mate  Sample 


Statistic 


Number  of  Callouts 


10  27  44 


LO  Score 

1.10 

1.60 

1.60 

Q1 

1.80 

3.80 

4.90 

02  (Median) 

2.15 

15.90 

3.00 

8.10 

23.45 

HI  Score 

6.20 

11.90 

63.00 

HI-LO 

5.10 

10.30 

61.40 

Q3-Q1 

1.20 

4.30 

18.55 

N 

14 

15 

15 

Group 

B 

A 

D 

Note.  H  -  22.17,  p  <  .001  (p  -  .001  for  H  >  16.27) 


Table  C-3 


Test  Item  Distribution  Statistics 


Task  1  (Point  to 

Part  Given 

Callout  Number) 

CROSS-SECTIONAL  VIEW: 

N/N-SEQ 

Sonar 

Technician 

Sample 

Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

1.00 

0.70 

1.10 

0.60 

Q1 

1.20 

1.65 

2.35 

2.00 

Q2  (Median) 

1.85 

2.80 

3.00 

2.40 

Q3 

2.40 

3.20 

3.70 

3.55 

HI  Score 

5.60 

5.40 

9.20 

9.30 

HI-LO 

4.60 

4.70 

8.10 

8.70 

Q3-Q1 

1.20 

1.55 

1.35 

1.55 

N 

36 

36 

36 

36 

Group 

C 

B 

A 

B 

Note.  H  -  20. 

14,  p  <  .001  (p 

*  .001  for  H 

[  >  16.27) 

Combined 

Boiler  Technician 

and  Gunner's 

Mate  Sample 

Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

1.10 

1.90 

1.80 

1.90 

Q1 

2.05 

2.80 

2.55 

3.00 

Q2  (Median) 

2.20 

3.15 

3.20 

3.50 

Q3 

3.25 

4.00 

4.05 

4.10 

HI  Score 

4.00 

7.00 

5.30 

8.30 

HI-LO 

2.90 

5.10 

3.50 

6.40 

Q3-Q1 

1.20 

1.20 

1.50 

1.10 

N 

15 

14 

15 

14 

Group 

C 

B 

A 

B 

Note.  H  *  8.04, 

p  <  .05  (p  *  .05 

for  H  >  7.82) 

C-3 


Table  C-4 


Test  Item  Distribution  Statistics 
Task  1  (Point  to  Part  Given  Callout  Number) 
CROSS-SECTIONAL  VIEW:  N/N-RAN 


Sonar  Technician  Sample 


Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

0.50 

0.60 

0.60 

1.20 

Q1 

1.20 

1.20 

3.95 

4.55 

02  (Median) 

2.00 

2.10 

6.00 

8.25 

Q3 

2.80 

3.55 

8.65 

10.95 

HI  Score 

6.40 

7.30 

31.40 

17.00 

HI-LO 

5.90 

6.70 

30.80 

15.80 

Q3-Q1 

1.60 

2.35 

4.70 

6.40 

N 

36 

36 

36 

36 

Group 

D 

C 

B 

C 

Note.  H  =  65.72, 

p  <  .001  (p  = 

.001  for  H  > 

16.27) 

Combined 

Boiler  Technician  and  Gunner's  Mate  Sample 

Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

1.00 

1.30 

1.60 

3.60 

Q1 

1.40 

1.95 

4.80 

7.70 

02  (Median) 

2.00 

2.90 

7.95 

10.60 

Q3 

2.70 

6.25 

14.30 

13.55 

HI  Score 

4.10 

7.60 

20.90 

25.20 

HI-LO 

3.10 

6.30 

19.30 

21.60 

Q3-Q1 

1.30 

4.30 

9.50 

5.85 

N 

15 

15 

14 

15 

Group 

D 

C 

B 

C 

Table  C-5 


Test  Item  Distribution  Statistics 
Task  1  (Point  to  Part  Given  Callout  Number) 
EXPLODED  VIEW:  SEQ/C/E 


Sonar  Technician  Sample 


Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

1.10 

1.10 

0.40 

0.80 

Q1 

1.80 

1.65 

1.10 

1.40 

Q2  (Median) 

2.00 

2.30 

1.25 

2.00 

Q3 

2.65 

3.10 

1.65 

2.75 

HI  Score 

5.00 

6.20 

4.50 

5.60 

HI-LO 

3.90 

5.10 

4.10 

4.80 

Q3-Q1 

0.85 

1.45 

0.55 

1.35 

N 

36 

36 

36 

36 

Group 

A 

D 

C 

D 

Note.  H  *  31.1, 

p  <  .001  (p  * 

.001  for  H 

>  16.27) 

Combined 

Boiler  Technician 

and  Gunner's  Mate  Sample 

Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

1.30 

2.20 

1.10 

1.90 

Q1 

1.90 

3.20 

1.50 

2.60 

Q2  (Median) 

2.90 

4.10 

2.00 

4.00 

Q3 

3.90 

6.55 

3.55 

6.95 

HI  Score 

10.90 

20.60 

5.10 

12.00 

HI-LO 

9.60 

18.40 

4.00 

10.10 

Q3-Q1 

2.00 

3.35 

2.05 

4.35 

N 

15 

15 

15 

15 

Group 

A 

D 

C 

D 

Note.  H  “  15.91 , 

p  <  .01  (p  »  .01 

for  H  > 

11.34) 

C-5 


Table  C-6 


Test  Item  Distribution  Statistics 
Task  1  (Point  to  Part  Given  Callout  Number) 
EXPLODED  VIEW:  SEQ/C/NE 


Sonar  Technician  Sample 


Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

0.50 

0.50 

0.40 

0.50 

Ql 

1.00 

1.20 

2.00 

1.15 

02  (Median) 

1.20 

1.90 

3.85 

1.60 

Q3 

2.00 

3.00 

5.70 

2.35 

HI  Score 

5.00 

9.00 

15.00 

4.80 

HI-LO 

A. 50 

8.50 

14.60 

4.30 

Q3-Q1 

1.00 

1.80 

3.70 

1.20 

N 

36 

36 

36 

36 

Group 

B 

A 

D 

C 

Note.  H  -  32.2, 

p  <  .001  (p  = 

.001  for  H 

>  16.27) 

Combined  Boiler  Technician  and  Gunner's  Mate  Sample 


Statistic 


Number  of  Callouts 


10 

27 

44 

62 

LO  Score 

1.10 

1.20 

Ql 

1.30 

1.30 

5.35 

1.15 

02  (Median) 

3.20 

8.00 

2.20 

Q3 

2.10 

10.45 

5.15 

HI  Score 

3.20 

15.80 

HI-LO 

2.20 

14.70 

26.80 

Q3-Q1 

0.80 

7.75 

5.10 

N 

14 

15 

15 

15 

Group 

B 

A 

D 

C 

Note.  H  -  17.92, 

p  <  .001  (p  • 

•  .001  for  H  > 

16.27) 

C-6 


Table  C-7 


Test  Item  Distribution  Statistics 
Task  1  (Point  to  Part  Given  Callout  Number) 
EXPLODED  VIEW:  SEQ/NC/E 


Sonar  Technician  Sample 


Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

0.90 

1.60 

1.20 

0.70 

Q1 

1.80 

2.30 

2.25 

1.20 

Q2  (Median) 

2.45 

3.00 

2.80 

1.95 

Q3 

3.10 

4.10 

3.25 

3.50 

HI  Score 

6.90 

8.60 

5.80 

8.00 

HI-LO 

6.00 

7.00 

4.60 

7.30 

Q3-Q1 

1.30 

1.80 

1.00 

2.30 

N 

36 

36 

36 

36 

Group 

C 

B 

A 

B 

Note.  H  *  12.87, 

p  <  .01  (p  = 

.01  for  H  > 

11.34) 

Comb ined 

Boiler  Technician  and  Gunner's  Mate  Sample 

Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

1.30 

1.90 

1.60 

1.30 

Q1 

2.15 

2.50 

2.75 

1.90 

Q2  (Median) 

2.60 

3.95 

3.10 

3.10 

Q3 

5.55 

5.10 

4.90 

4.60 

HI  Score 

12.10 

9.80 

11.20 

9.00 

HI-LO 

10.80 

7.90 

9.60 

7.70 

Q3-Q1 

3.40 

2.60 

2.15 

2.70 

N 

15 

14 

15 

14 

Group 

C 

B 

A 

B 

Note.  H  -  1.75, 

p  >  .05  (p  *  . 

05  for  H  >  7.82) 

C-7 
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Table  C-8 


Test  Item  Distribution  Statistics 
Task  1  (Point  to  Part  Given  Callout  Number) 
EXPLODED  VIEW:  SEQ/NC/NE 


Sonar  Technician  Sample 


Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

0.60 

1.00 

0.90 

0.50 

Q1 

1.30 

2.00 

1.20 

2.60 

02  (Median) 

2.20 

2.95 

1.85 

4.65 

Q3 

3.85 

5.20 

2.90 

7.05 

HI  Score 

9.00 

26.20 

7.10 

12.50 

HI-LO 

8.40 

25.20 

6.20 

12.00 

Q3-Q1 

2.55 

3.20 

1.70 

4.45 

N 

36 

36 

36 

36 

Group 

D 

C 

B 

A 

Note.  H  *  22.96, 

p  <  .001  (p  * 

.001  for  H 

>  16.27) 

Combined  Boiler  Technician 

and  Gunner ' s 

Mate  Sample 

Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

1.30 

1.30 

0.80 

2.20 

Q1 

2.60 

3.55 

1.90 

02  (Median) 

3.80 

KO 

2.50 

5.00 

8.65 

Ksa 

5.00 

7.60 

HI  Score 

17.70 

BBS 

7.20 

18.00 

HI-LO 

16.40 

31.60 

6.40 

15.80 

Q3-Q1 

6.05 

3.45 

3.10 

4.60 

N 

15 

15 

14 

15 

Group 

D 

C 

B 

A 

Note.  H  *  7.77, 

p  >  .05  (p  ■  .05 

for  H  > 

7.82) 

C-8 


Table  C-9 


1  < 
f  » 

i  1 

1;' ' 

1  ‘ 
[; 

Test  Item  Distribution  Statistics 

Task  1  (Point  to  Part  Given  Callout  Number) 
EXPLODED  VIEW:  RAN/C/E 

Sonar  Technician  Sample 

j 

[  i 

Statistic 

Number 

of  Callouts 

i : 

i  • 

t  « 

i 

10 

27 

44 

62 

I 

i 

LO  Score 

0.90 

0.80 

2.30 

1.70 

»  . 

Q1 

2.00 

2.05 

2.95 

4.65 

E  > 

r  i 

Q2  (Median) 

2.20 

3.40 

3.95 

8.55 

:■ 

Q3 

2.95 

6.25 

5.05 

16.10 

|  i 

E  ! 

HI  Score 

7.50 

15.80 

13.80 

24.20 

r  ! 

HI-LO 

6.60 

15.00 

11.50 

22.50 

p  ( 

Q3-Q1 

0.95 

4.20 

2.10 

11.45 

N 

36 

36 

36 

36 

Group 

D 

C 

B 

C 

E 

Note.  H  -  48.43, 

p  <  .001  (p  * 

.001  for  H 

>  16. 

27) 

[j 

Combined 

Boiler  Technician  and  Gunner's 

Mate  Sample 

l  ' 

Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

1.30 

1.70 

2.10 

5.00 

• 

Q1 

2.45 

4.00 

3.80 

7.35 

Q2  (Median) 

3.00 

5.10 

5.55 

13.90 

Q3 

4.05 

15.70 

7.50 

35.80 

• 

HI  Score 

6.80 

32.00 

19.80 

60.80 

HI-LO 

5.50 

30.30 

17.70 

55.80 

’ 

Q3-Q1 

1.60 

11.70 

3.70 

28.45 

I  1 

N 

15 

15 

14 

15 

Group 

D 

C 

B 

C 

r ' 

Note.  H  -  25.72, 

p  <  .001  (p  ■ 

.001  for  H 

>  16. 

27) 

Table  C-10 


Test  Item  Distribution  Statistics 
Task  1  (Point  to  Part  Given  Callout  Number) 
EXPLODED  VIEW:  RAN/C/NE 


Sonar  Technician  Sample 


Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

0.90 

1.00 

2.40 

0.60 

Q1 

1.80 

2.35 

5.50 

6.25 

02  (Median) 

2.55 

3.30 

7.85 

9.85 

Q3 

3.35 

5.00 

13.35 

18.15 

HI  Score 

6.00 

7.60 

51.20 

52.20 

HI-LO 

5.10 

6.60 

48.80 

51.60 

Q3-Q1 

1.55 

2.65 

7.85 

11.90 

N 

36 

36 

36 

36 

Group 

C 

B 

A 

B 

Note.  H  =  74.36, 

p  <  .001  (p  = 

!  .001  for  H  > 

16.27) 

Combined 

Boiler  Technician 

and  Gunner 

1  s  Mate  Sample 

Statistic 

Number  of 

Callouts 

10 

27 

44 

62 

LO  Score 

1.70 

2.10 

1.40 

1.30 

Ql 

2.80 

3.70 

5.55 

8.10 

02  (Median) 

3.20 

4.05 

8.10 

10.85 

Q3 

5.55 

5.10 

17.05 

15.30 

HI  Score 

8.00 

9.10 

50.10 

43.80 

HI-LO 

6.30 

7.00 

48.70 

42.50 

Q3-Q1 

2.75 

1.40 

11.50 

7.20 

N 

15 

14 

15 

14 

Group 

C 

B 

A 

B 

Note.  H  -  20.82, 

p  <  .001  (p  -  .001  for  H  > 

16.27) 

C-10 


Table  C-ll 


Test  Item  Distribution  Statistics 
Task  1  (Point  to  Part  Given  Callout  Number) 
EXPLODED  VIEW:  RAN/NC/E 


Sonar  Technician  Sample 


Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

1.30 

1.50 

2.00 

2.10 

Q1 

2.90 

2.20 

4.10 

4.70 

Q2  (Median) 

4.15 

3.20 

6.95 

6.40 

Q3 

5.55 

4.60 

12.95 

10.95 

HI  Score 

23.60 

11.10 

26.50 

35.20 

HI-LO 

22.30 

9.60 

24.50 

33.10 

Q3-Q1 

2.65 

2.40 

8.85 

6.25 

N 

36 

36 

36 

36 

Group 

B 

A 

D 

A 

Note.  H  -  31.16, 

p  <  .001  (p  * 

■  .001  for  H  > 

16.27) 

Combined 

Boiler  Technician 

and  Gunner 

'  s  Mate  Sample 

Statistic 

Number  of 

Callouts 

10 

27 

44 

62 

LO  Score 

1.90 

1.20 

4.00 

2.00 

Q1 

2.80 

2.80 

6.65 

6.20 

02  (Median) 

3.55 

3.70 

7.90 

11.60 

Q3 

5.90 

7.50 

14.70 

22.20 

HI  Score 

34.20 

11.40 

16.50 

36.90 

HI-LO 

32.30 

10.20 

12.50 

34.90 

Q3-Q1 

3.10 

4.70 

8.05 

16.00 

N 

14 

15 

15 

15 

Group 

B 

A 

D 

A 

Note.  H  -  16.42, 

p  <  .001  (p  *  .001  for  H  > 

16.27) 

C-ll 


Table  C-12 

Test  Item  Distribution  Statistics 
Task  1  (Point  to  Part  Given  Callout  Number) 
EXPLODED  VIEW:  RAN/NC/NE 


Sonar  Technician  Sample 


Statistic 

Number  of 

Callouts 

10 

27 

44 

62 

LO  Score 

0.60 

0.80 

1.40 

1.60 

Ql 

1.25 

2.55 

4.80 

3.00 

Q2  (Median) 

1.85 

4.85 

9.20 

6.00 

Q3 

HI  Score 

2.80 

8.35 

11.20 

10.70 

5.00 

46.00 

28.30 

21.20 

HI-LO 

4.40 

45.20 

26.90 

19.60 

Q3-Q1 

1.55 

5.80 

6.40 

7.70 

N 

36 

36 

36 

36 

Group 

A 

D 

C 

D 

Note.  H  *  52.18, 

p  <  .001  (p  * 

.001  for  H  > 

16.27) 

Combined 

Boiler  Technician  and  Gunner 

'  s  Mate  Sample 

Statistic 

Number  of 

Callouts 

10 

27 

44 

62 

LO  Score 

1.10 

2.10 

5.00 

2.30 

Ql 

Q2  (Median) 

2.05 

3.35 

8.60 

4.90 

3.00 

6.10 

11.50 

12.30 

Q3 

HI  Score 

4.90 

21.45 

22.20 

17.95 

11.00 

36.50 

37.10 

51.90 

HI-LO 

9.90 

34.40 

32.10 

49.60 

Q3-Q1 

2.85 

18.10 

13.60 

13.05 

N 

15 

15 

15 

15 

Group 

A 

D 

C 

D 

Note.  H  ■  20.61, 

p  <  .001  (p  ■ 

.001  for  H  > 

16.27) 

C-12 


Table  C-13 


Test  Item  Distribution  Statistics 
Task  2  (Point  to  Part  Given  Nomenclature) 
CROSS-SECTIONAL  VIEW:  NOMEN 


Sonar  Technician  Sample 


i 


Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

0.90 

1.40 

1.60 

1.20 

Ql 

1.10 

3.90 

2.95 

4.75 

Q2  (Median) 

1.55 

6.20 

5.80 

8.50 

Q3 

2.45 

8.90 

11.05 

14.75 

HI  Score 

6.50 

19.50 

36.30 

40.30 

HI-LO 

5.60 

18.10 

34.70 

39.10 

Q3-Q1 

1.35 

5.00 

8.10 

10.00 

N 

36 

36 

36 

36 

Group 

D 

A 

B 

C 

Note.  H  -  59.93, 

p  <  .001  (p  ■ 

.001  for  H 

>  16.27) 

Combined 

Boiler  Technician 

and  Gunner's  Mate  Sample 

Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

2.00 

2.10 

1.80 

2.00 

Ql 

2.05 

4.85 

4.00 

3.80 

Q2  (Median) 

2.30 

9.90 

6.95 

6.10 

Q3 

4.20 

12.60 

8.60 

17.25 

HI  Score 

8.60 

18.10 

27.20 

28.10 

HI-LO 

6.60 

16.00 

25.40 

26.10 

Q3-Q1 

2.15 

7.75 

4.60 

13.45 

N 

15 

15 

14 

15 

Group 

D 

A 

B 

C 

Note.  H  -  15.97, 

p  <  .01  (p  ■  .01 

for  H  > 

11.34) 

C-13 


Statistic 


LO 

Score 

Q1 

Q2 

(Median) 

Q3 

HI 

Score 

HI- 

-LO 

Table  C-14 

Test  Item  Distribution  Statistics 
Task  2  (Point  to  Part  Given  Nomenclature) 
CROSS-SECTIONAL  VIEW:  NUM-SEQ 


Sonar  Technician  Sample 


Number  of  Callouts 


2 


0.50 

0.50 

0.50 

0.60 

0.90 

2.10 

0.55 

2.70 

2.05 

3.40 

2.85 

4.70 

2.40 

6.65 

4.35 

7.05 

8.60 

19.50 

15.10 

23.90 

8.10 

19.00 

14.60 

23.30 

1  .SO 

4.55 

3.80 

4.35 

36  36  36  36 

B  C  D  A 

27.52,  p  <  .001  (p  «  .001  for  H  >  16.27) 


Combined  Boiler  Technician  and  Gunner's  Mate  Sample 


Statistic 


Number  of  Callouts 


2 


LO  Score 

1.20 

0 

Q1 

1.50 

3 

Q2  (Median) 

2.25 

5 

Q3 

2.90 

8 

HI  Score 

5.00 

10 

HI-LO 

3.80 

10 

Q3-Q1 

1.40 

4 

N 

14 

Group 

B 

2.90 

3.90 

6.00 

15.55 

23.20 

20.30 

11.65 


p  <  .001  (p  -  .001  for  H  >  16.27) 


Table  C-15 


Test  Item  Distribution  Statistics 
Task  2  (Point  to  Part  Given  Nomenclature) 
CROSS-SECTIONAL  VIEW:  NUM-RAN 


Sonar  Technician  Sample 


Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

0.50 

0.50 

0.50 

0.90 

Q1 

0.80 

2.15 

5.80 

7.70 

Q2  (Median) 

1.60 

4.40 

9.25 

11.60 

Q3 

2.10 

6.75 

13.05 

13.20 

HI  Score 

5.30 

19.30 

25.60 

36.20 

HI-LO 

4.80 

18.80 

25.10 

35.30 

Q3-Q1 

1.30 

4.60 

7.25 

5.50 

N 

36 

36 

36 

36 

Group 

C 

D 

A 

B 

Note.  H  *  70.71, 

p  <  .001  (p  * 

.001  for  H 

>  16.27) 

Combined 

Boiler  Technician  and  Gunner's  Mate  Sample 

Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

1.10 

0.50 

0.70 

2.70 

Q1 

1.45 

2.15 

3.25 

8.10 

Q2  (Median) 

2.80 

5.70 

9.60 

11.60 

Q3 

4.20 

9.60 

12.90 

18.10 

HI  Score 

4.90 

20.10 

21.50 

24.70 

HI-LO 

3.80 

19.60 

20.80 

22.00 

Q3-Q1 

2.75 

7.45 

9.65 

10.00 

N 

15 

15 

15 

14 

Group 

C 

D 

A 

B 

Note.  H  ■  20,  p 

<  .001  (p  -  .001  for  H  >  16. 

.27) 

C-15 


Table  C-16 


Test  Item  Distribution  Statistics 
Task  2  (Point  to  Part  Given  Nomenclature) 
CROSS-SECTIONAL  VIEW:  N/N-SEQ 


Sonar  Technician  Sample 


Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

0.50 

2.30 

1.30 

1.80 

Q1 

1.30 

5.90 

6.00 

7.00 

02  (Median) 

2.30 

8.05 

10.25 

13.25 

Q3 

3.00 

11.90 

15.15 

17.05 

HI  Score 

4.10 

104.00 

38.00 

46.00 

HI-LO 

3.60 

101.70 

36.70 

44.20 

Q3-Q1 

1.70 

6.00 

9.15 

10.05 

N 

36 

36 

36 

36 

Group 

D 

A 

B 

C 

Note.  H  =  70.92, 

p  <  .001  (p 

*  .001  for  H  > 

16.27) 

Combined 

Boiler  Technician 

and  Gunner 

'  s  Mate  Sample 

Statistic 

Number  of 

Callouts 

10 

27 

44 

62 

LO  Score 

1.50 

1.00 

5.70 

2.20 

Q1 

2.20 

7.30 

6.70 

9.25 

02  (Median) 

3.00 

13.10 

9.15 

15.40 

Q3 

3.90 

24.40 

14.00 

30.45 

HI  Score 

4.80 

46.00 

21.90 

66.80 

HI-LO 

3.30 

45.00 

16.20 

64.60 

Q3-Q1 

1.70 

17.10 

7.30 

21.20 

N 

15 

15 

14 

15 

Group 

D 

A 

B 

C 

Note.  H  -  26.92, 

p  <  .001  (p  ■  .001  for  H  > 

16.27) 

C-16 


Table  C-17 


Test  Item  Distribution  Statistics 
Task  2  (Point  to  Part  Given  Nomenclature) 
CROSS-SECTIONAL  VIEW:  N/N-RAN 


Sonar  Technician  Sample 


Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

1.00 

0.90 

1.50 

2.50 

Q1 

1.45 

3.25 

3.50 

6.85 

Q2  (Median) 

2.05 

4.70 

5.45 

11.75 

Q3 

3.00 

7.65 

8.55 

26.95 

HI  Score 

4.20 

14.30 

21.00 

63.10 

HI-LO 

3.20 

13.40 

19.50 

60.60 

Q3-Q1 

1.55 

4.40 

5.05 

20.10 

N 

36 

36 

36 

36 

Group 

A 

B 

C 

D 

Note.  H  -  78.19, 

p  <  .001  (p  ■ 

.001  for  H 

>  16.27) 

Combined 

Boiler  Technician 

and  Gunner's  Mate  Sample 

Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

1.20 

1.70 

2.30 

1.90 

Q1 

2.10 

3.60 

4.90 

17.25 

Q2  (Median) 

2.70 

6.95 

10.10 

31.00 

Q3 

3.40 

7.80 

17.45 

50.25 

HI  Score 

5.50 

16.10 

27.10 

149.00 

HI-LO 

4.30 

14.40 

24.80 

147.10 

Q3-Q1 

1.30 

4.20 

12.55 

33.00 

N 

15 

14 

15 

15 

Group 

A 

B 

C 

D 

Note.  H  -  31.74, 

p  <  .001  (p  ■  .001  for  H 

>  16.27) 

C-17 


r" 


__ 


f 

!►" 

I 

I 


I 

I 

I  1 


. 


I  * 

\  f 


Table  C-19 

Test  Item  Distribution  Statistics 
Task  2  (Point  to  Part  Given  Nomenclature) 
EXPLODED  VIEW:  SEQ/C/NE 


Sonar  Technician  Sample 


Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

0.50 

0.50 

0.50 

0.50 

Q1 

0.50 

0.50 

1.10 

0.50 

Q2  (Median) 

1.15 

2.60 

3.00 

4.05 

Q3 

2.00 

4.45 

5.85 

6.40 

HI  Score 

3.00 

27.50 

14.40 

41.10 

HI-LO 

2.50 

27.00 

13.90 

40.60 

Q3-Q1 

1.50 

3.95 

4.75 

5.90 

N 

36 

36 

36 

36 

Group 

C 

B 

A 

D 

Note.  H  =*  15.03, 

p  <  .01  (p  * 

.01  for  H  >  11.34) 

I 

Combined 

Boiler  Technician  and  Gunner's  Mate  Sample 

Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

0.50 

0.50 

0.50 

0.50 

Q1 

1.50 

0.50 

2.00 

0.70 

Q2  (Median) 

2.10 

2.00 

4.20 

1.80 

Q3 

2.95 

3.60 

7.25 

11.95 

HI  Score 

4.10 

5.70 

10.30 

19.10 

HI-LO 

3.60 

5.20 

9.80 

18.60 

Q3-Q1 

1.45 

3.10 

5.25 

11.25 

N 

15 

14 

15 

15 

Group 

C 

B 

A 

D 

Note .  H  *  4.45,  p  >  .05  (p  ■  .05  for  H  >  7.82) 


Table  C-20 


Test  Item  Distribution  Statistics 

Task  2  (Point  to  Part  Given  Nomenclature) 

EXPLODED  VIEW:  SEQ/NC/E 

Sonar  Technician  Sample 

. 

: 

• 

Statistic 

Number  of 

Callouts 

■ 

; 

10 

27 

44 

62 

LO  Score 

0.50 

0.50 

1.00 

0.50 

Q1 

0.75 

0.90 

2.85 

0.95 

Q2  (Median) 

1.35 

2.80 

8.20 

3.10 

Q3 

2.65 

5.00 

12.60 

5.85 

HI  Score 

5.70 

14.40 

20.40 

19.40 

HI-LO 

5.20 

13.90 

19.40 

18.90 

Q3-Q1 

1.90 

4.10 

9.75 

4.90 

N 

36 

36 

36 

36 

Group 

D 

C 

B 

A 

I  Note.  H  =  29.9 

I 

,  p  <  .001  (p 

=  .001  for  H  >  16 

.27) 

! 

Combined  Boiler  Technician  and  Gunner' 

i 

s  Mate  Sample 

Statistic 

Number  of 

Callouts 

10 

27 

44 

62 

LO  Score 

1.30 

0.50 

0.60 

0.50 

Q1 

1.75 

2.90 

1.60 

3.10 

Q2  (Median) 

2.70 

3.40 

2.45 

4.00 

Q3 

5.95 

4.30 

9.60 

4.60 

HI  Score 

12.90 

11.40 

12.30 

13.20 

HI-LO 

11.60 

10.90 

11.70 

12.70 

Q3-Q1 

4.20 

1.40 

8.00 

1.50 

N 

15 

15 

14 

15 

Group 

D 

C 

B 

A 

Note.  H  -  .96, 

p  >  .05  (p  * 

.05  for  H  >  7.82) 

C-20 


Table  C-21 


Test  Item  Distribution  Statistics 
Task  2  (Point  to  Part  Given  Nomenclature) 
EXPLODED  VIEW:  SEQ/NC/NE 


Sonar  Technician  Sample 


Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

0.50 

0.50 

0.50 

0.50 

Q1 

1.20 

0.50 

1.10 

0.50 

Q2  (Median) 

2.20 

0.95 

1.70 

1.25 

Q3 

3.50 

5.45 

2.90 

8.60 

HI  Score 

18.10 

18.10 

16.20 

22.90 

HI-LO 

17.60 

17.60 

15.70 

22.40 

Q3-Q1 

2.30 

4.95 

1.80 

8.10 

N 

36 

36 

36 

36 

Group 

A 

D 

C 

B 

Note.  H  “  1.95, 

p  >  .05  (p  *  . 

,05  for  H  >  7.82) 

Combined 

Boiler  Technician 

and  Gunner's  Mate  Sample 

Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

0.50 

0.50 

2.10 

0.50 

Q1 

1.00 

0.50 

3.20 

5.00 

Q2  (Median) 

2.40 

2.20 

4.20 

7.35 

Q3 

4.35 

7.70 

5.55 

17.30 

HI  Score 

10.40 

15.30 

9.30 

22.60 

HI-LO 

9.90 

14.80 

7.20 

22.10 

Q3-Q1 

3.35 

7.20 

2.35 

12.30 

N 

15 

15 

15 

14 

Group 

A 

D 

C 

B 

Note.  H  =  10.96, 

p  <  .05  (p  *  .05 

for  H  > 

7.82) 

C-21 


Table  C-22 


Test  Item  Distribution  Statistics 
Task  2  (Point  to  Part  Given  Nomenclature) 
EXPLODED  VIEW:  RAN/C/E 


Sonar  Technician  Sample 


Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

0.50 

0.50 

0.50 

0.50 

Ql 

1.10 

3.55 

4.40 

0.85 

02  (Median) 

2.00 

6.20 

8.60 

3.70 

Q3 

2.75 

8.35 

16.10 

8.85 

HI  Score 

5.50 

18.60 

28.90 

23.10 

HI-LO 

5.00 

18.10 

28.40 

22.60 

Q3-Q1 

1.65 

4.80 

11.70 

8.00 

N 

36 

36 

36 

36 

Group 

A 

D 

C 

B 

Note.  H  =  42.69, 

p  <  .001  (p  = 

.001  for  H  > 

16.27) 

Combined 

Boiler  Technician 

and  Gunner's  Mate  Sample 

Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

0.50 

0.50 

5.10 

0.50 

Ql 

1.10 

4.95 

9.20 

1.80 

02  (Median) 

2.30 

15.00 

7.75 

Q3 

3.70 

16.75 

27.00 

9.80 

HI  Score 

6.70 

55.60 

55.50 

14.60 

HI-LO 

6.20 

55.10 

50.40 

14.10 

Q3-Q1 

2.60 

11.80 

17.80 

8.00 

N 

15 

15 

15 

14 

Group 

A 

D 

C 

B 

Note.  H  *  24.7  , 

>  16.27) 

C-22 


WPP 


Table  C-23 


Test  Item  Distribution  Statistics 
Task  2  (Point  to  Part  Given  Nomenclature) 
EXPLODED  VIEW:  RAN/C/NE 


Sonar  Technician  Sample 


Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

0.50 

0.50 

0.50 

0.50 

Q1 

0.50 

3.30 

1.50 

0.50 

Q2  (Median) 

1.30 

4.75 

7.85 

1.30 

Q3 

2.45 

6.45 

14.00 

4.30 

HI  Score 

7.40 

29.50 

54.30 

19.00 

HI-LO 

6.90 

29.00 

53.80 

18.50 

Q3-Q1 

1.95 

3.15 

12.50 

3.80 

N 

36 

36 

36 

36 

Group 

D 

C 

B 

A 

Note.  H  =  31.44, 

p  <  .001  (p  * 

.001  for  H 

>  16.27) 

Combined 

Boiler  Technician 

and  Gunner 1 s 

Mate  Sample 

Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

0.50 

0.50 

0.50 

0.50 

Q1 

1.20 

3.30 

1.10 

2.50 

Q2  (Median) 

4.40 

5.20 

6.00 

10.60 

Q3 

5.35 

6.40 

16.40 

14.05 

HI  Score 

21.20 

21.90 

33.90 

18.80 

HI-LO 

20.70 

21.40 

33.40 

18.30 

Q3-Q1 

4.15 

3.10 

15.30 

11.55 

N 

15 

15 

14 

15 

Group 

D 

C 

B 

A 

Note.  H  *  2.08, 

p  >  .05  (p  *  .05 

for  H  >  7.82) 

C-23 


Table  C-24 


Test  Item  Distribution  Statistics 
Task  2  (Point  to  Part  Given  Nomenclature) 
EXPLODED  VIEW:  RAN/NC/E 


Sonar  Technician  Sample 


Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

1.80 

0.50 

0.50 

0.50 

Q1 

3.20 

2.40 

9.05 

7.70 

Q2  (Median) 

4.10 

6.05 

18.30 

11.45 

Q3 

5.80 

9.05 

25.60 

23.70 

HI  Score 

27.20 

40.60 

73.90 

56.30 

HI-LO 

25.40 

40.10 

73.40 

55.80 

Q3-Q1 

2.60 

6.65 

16.55 

16.00 

N 

36 

36 

36 

36 

Group 

C 

B 

A 

D 

Note.  H  =  42.4, 

p  <  .001  (p  * 

.001  for  H 

>  16.27) 

Combined 

Boiler  Technician  and  Gunner 

'  8  Mate  Sample 

Statistic 

Number  of 

Callouts 

10 

27 

44 

62 

LO  Score 

2.80 

1.40 

2.70 

2.40 

Ql 

4.45 

2.90 

10.00 

17.05 

02  (Median) 

6.50 

5.30 

17.10 

27.80 

Q3 

16.00 

9.50 

36.75 

56.10 

HI  Score 

21.10 

15.30 

51.60 

70.50 

HI-LO 

18.30 

13.90 

48.90 

68.10 

Q3-Q1 

11.55 

6.60 

26.75 

39.05 

N 

15 

14 

15 

15 

Group 

C 

B 

A 

D 

Note.  H  -  23.55, 

p  <  .001  (p  - 

.001  for  H  > 

16.27) 

C-24 


Table  C-25 


Test  Item  Distribution  Statistics 
Task  2  (Point  to  Part  Given  Nomenclature) 
EXPLODED  VIEW:  RAN/NC/NE 


Sonar  Technician  Sample 


Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

0.50 

0.50 

0.50 

0.50 

Q1 

0.50 

0.95 

5.55 

5.25 

Q2  (Median) 

1.20 

2.90 

8.80 

13.90 

Q3 

2.60 

5.95 

13.50 

27.65 

HI  Score 

6.70 

9.90 

28.10 

53.80 

HI-LO 

6.20 

9.40 

27.60 

53.30 

Q3-Q1 

2.10 

5.00 

7.95 

22.40 

N 

36 

36 

36 

36 

Group 

B 

A 

D 

C 

Note.  H  -  60.91, 

p  <  .001  (p  = 

.001  for  H  > 

16.27) 

Combined 

Boiler  Technician 

and  Gunner's  Mate  Sample 

Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

0.50 

0.60 

3.20 

4.20 

Q1 

0.80 

1.85 

7.95 

11.60 

02  (Median) 

1.15 

7.70 

9.60 

14.30 

Q3 

2.00 

14.95 

24.60 

39.15 

HI  Score 

6.00 

20.60 

72.00 

106.30 

HI-LO 

5.50 

20.00 

68.80 

102.10 

Q3-Q1 

1.20 

13.10 

16.65 

27.55 

N 

14 

15 

15 

15 

Group 

B 

A 

D 

C 

Note.  H  ■  30.51, 

p  <  .001  (p  ■  .001  for  H  > 

16.27) 

C-25 


Table  C-26 


Test  Item  Distribution  Statistics 
Task  3  (Tell  Nomenclature  of  Marked  Part) 
CROSS-SECTIONAL  VIEW:  NOMEN 


Sonar  Technician  Sample 


Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

0.50 

0.80 

0.80 

1.00 

Ql 

0.80 

1.20 

2.00 

1.70 

02  (Median) 

1.20 

1.30 

2.25 

2.00 

Q3 

1.50 

1.95 

3.00 

2.60 

HI  Score 

2.20 

3.60 

8.10 

11.10 

HI-LO 

1.70 

2.80 

7.30 

10.10 

Q3-Q1 

0.70 

0.75 

1.00 

0.90 

N 

36 

36 

36 

36 

Group 

B 

C 

D 

A 

Note.  H  =  53.94, 

p  <  .001  (p  “ 

.001  for  H 

>  16.27) 

Combined 

Boiler  Technician  and  Gunner's  Mate  Sample 

Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

1.00 

1.00 

2.20 

1.80 

Ql 

1.10 

1.90 

2.70 

2.05 

02  (Median) 

1.50 

2.10 

3.60 

2.60 

Q3 

2.30 

2.75 

4.20 

3.55 

HI  Score 

3.20 

6.90 

6.30 

4.60 

HI-LO 

2.20 

5.90 

4.10 

2.80 

Q3-Q1 

1.20 

0.85 

1.50 

1.50 

N 

14 

15 

15 

15 

Group 

B 

C 

D 

A 

Note.  H  -  21.56, 

p  <  .001  (p  ■ 

.001  for  H 

>  16.27) 

C-26 


Table  C-27 


Test  Item  Distribution  Statistics 
Task  3  (Tell  Nomenclature  of  Marked  Part) 
CROSS-SECTIONAL  VIEW:  NUM-SEQ 


Sonar  Technician  Sample 


Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

1.10 

2.30 

2.00 

4.00 

Q1 

1.80 

3.65 

3.00 

5.30 

Q2  (Median) 

2.15 

5.00 

3.35 

6.05 

Q3 

3.00 

7.60 

4.25 

7.25 

HI  Score 

6.00 

15.70 

6.60 

13.00 

HI-LO 

4.90 

13.40 

4.60 

9.00 

Q3-Q1 

1.20 

3.95 

1.25 

1.95 

N 

36 

36 

36 

36 

Group 

C 

D 

A 

B 

Note.  H  -  89.34, 

p  <  .001  (p  = 

.001  for  H 

>  16.27) 

Combined 

Boiler  Technician 

and  Gunner 

1  s  Mate  Sample 

Statistic 

Number  of 

Callouts 

10 

27 

44 

62 

LO  Score 

1.90 

3.30 

4.00 

4.90 

Q1 

2.90 

4.25 

6.10 

Q2  (Median) 

3.10 

4.90 

7.55 

Q3 

3.80 

6.00 

9.00 

HI  Score 

5.00 

9.60 

28.10 

HI-LO 

3.10 

7.30 

5.60 

23.20 

Q3-Q1 

0.90 

1.75 

2.90 

N 

15 

15 

15 

14 

Group 

C 

D 

A 

B 

Note.  H  *  33.06, 

p  <  .001  (p  ■  .001  for  H  > 

16.27) 

C-27 


Table  C-28 

Test  Item  Distribution  Statistics 
Task  3  (Tell  Nomenclature  of  Marked  Part) 
CROSS-SECTIONAL  VIEW:  NUM-RAN 


Sonar  Technician  Sample 


Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

1.30 

2.80 

1.80 

2.70 

Q1 

2.00 

3.80 

2.80 

4.90 

Q2  (Median) 

2.20 

4.60 

3.70 

5.75 

Q3 

2.80 

5.20 

4.45 

7.05 

HI  Score 

3.80 

7.00 

6.20 

10.20 

HI-LO 

2.50 

4.20 

4.40 

7.50 

Q3-Q1 

0.80 

1.40 

1.65 

2.15 

N 

36 

36 

36 

36 

Group 

D 

A 

B 

C 

Note.  H  -  84.07, 

p  <  .001  (p  - 

.001  for  H 

>  16.27) 

Combined 

Boiler  Technician 

and  Gunner's  Mate  Sample 

Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

2.20 

3.20 

2.90 

4.20 

Q1 

2.95 

4.45 

3.60 

5.15 

Q2  (Median) 

3.30 

5.60 

4.15 

6.00 

Q3 

4.15 

7.45 

4.90 

7.10 

HI  Score 

14.00 

9.10 

6.90 

9.30 

HI-LO 

11.80 

5.90 

4.00 

5.10 

Q3-Q1 

1.20 

3.00 

1.30 

1.95 

N 

15 

15 

14 

15 

Group 

D 

A 

B 

C 

Note.  H  -  24.41, 

p  <  .001  (p  ■  .001  for  H 

>  16.27) 

C-28 


Table  C-29 


Test  Item  Distribution  Statistics 
Task  3  (Tell  Nomenclature  of  Marked  Part) 
CROSS-SECTIONAL  VIEW:  N/N-SEQ 


Sonar  Technician  Sample 


t 


Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

1.00 

0.80 

0.50 

1.00 

Q1 

1.50 

1.45 

1.35 

1.70 

02  (Median) 

2.00 

1.80 

2.00 

2.05 

Q3 

2.20 

2.60 

2.60 

2.65 

HI  Score 

4.00 

4.80 

11.10 

3.60 

HI-LO 

3.00 

4.00 

10.60 

2.60 

Q3-Q1 

0.70 

1.15 

1.25 

0.95 

N 

36 

36 

36 

36 

Group 

A 

B 

C 

D 

Note.  H  =  2.4, 

p  >  .05  (p  =  .05  for  H  >  7.82) 

Combined 

Boiler  Technician 

and  Gunner's 

Mate  Sample 

Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

1.70 

1.80 

1.10 

1.50 

Q1 

2.20 

2.30 

2.85 

3.00 

3.05 

4.55 

3.35 

HI  Score 

5.20 

4.80 

HI-LO 

3.50 

3.00 

9.60 

K9 

Q3-Q1 

1.05 

1.70 

2.75 

1.15 

N 

15 

14 

15 

15 

Group 

A 

B 

C 

D 

Note.  H  -  3.69, 

p  >  .05  (p  ■  .05 

for  H  >  7.82) 
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Table  C-30 


Test  Item  Distribution  Statistics 
Task  3  (Tell  Nomenclature  of  Marked  Part) 
CROSS-SECTIONAL  VIEW:  N/N-RAN 


Sonar  Technician  Sample 


Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

0.80 

0.80 

1.00 

1.70 

Ql 

1.90 

1.20 

2.05 

2.85 

Q2  (Median) 

2.30 

1.85 

2.75 

3.65 

Q3 

2.95 

2.40 

4.25 

4.50 

HI  Score 

8.60 

4.00 

12.70 

12.00 

HI-LO 

7.80 

3.20 

11.70 

10.30 

Q3-Q1 

1.05 

1.20 

2.20 

1.65 

N 

36 

36 

36 

36 

Group 

B 

C 

D 

A 

Note.  H  =  37.45, 

p  <  .001  (p  ■ 

.001  for  H  > 

16.27) 

Combined 

Boiler  Technician 

and  Gunner 

'  s  Mate  Sample 

Statistic 

Number  of 

Callouts 

10 

27 

44 

62 

LO  Score 

2.10 

1.60 

2.50 

1.90 

Ql 

2.30 

2.20 

3.05 

3.50 

02  (Median) 

3.00 

2.30 

3.40 

4.10 

Q3 

4.10 

2.55 

4.70 

5.55 

HI  Score 

5.10 

3.90 

6.50 

7.00 

HI-LO 

3.00 

2.30 

4.00 

5.10 

Q3-Q1 

1.80 

0.35 

1.65 

2.05 

N 

14 

15 

15 

15 

Group 

B 

C 

D 

A 

Note.  H  -  24.04, 

p  <  .001  (p  ■  .001  for  H  > 

16.27) 

Table  C-31 


Test  Item  Distribution  Statistics 
Task  3  (Tell  Nomenclature  of  Marked  Part) 
EXPLODED  VIEW:  SEQ/C/E 


Sonar  Technician  Sample 


LO  Score 

1.40 

1.70 

2.1 

Q1 

2.05 

3.05 

3.1 

Q2  (Median) 

2.60 

3.60 

3.2 

Q3 

3.00 

4.25 

4.0 

HI  Score 

3.80 

10.00 

10.7 

HI-LO 

2.40 

8.30 

8.6 

Q3-Q1 

0.95 

1.20 

0.9 

N 

36 

36 

36 

Group 

C 

B 

A 

69.68,  p  <  .001  (p  =  .001  for  H  >  16.27) 


Combined  Boiler  Technician  and  Gunner's  Mate  Sample 


Statistic 


Number  of  Callouts 


2 


LO  Score 

2.50 

Q1 

2.95 

Q2  (Median) 

3.00 

Q3 

3.90 

HI  Score 

6.30 

HI-LO 

3.80 

Q3-Q1 

0.95 

N 

15 

Group 

C 

p  <  .001  (p  -  .001  for  H  >  16.27) 


i 

! 


Table  C-32 

Test  Item  Distribution  Statistics 
Task  3  (Tell  Nomenclature  of  Marked  Part) 
EXPLODED  VIEW:  SEQ/C/NE 


Sonar  Technician  Sample 


Statistic 

Number 

of  Callouts 

10 

27 

44 

LO  Score 

1.60 

1.80 

2.00 

Q1 

2.25 

2.15 

2.80 

02  (Median) 

2.95 

2.45 

3.20 

Q3 

3.60 

3.00 

3.60 

HI  Score 

5.00 

3.80 

4.10 

HI-LO 

3.40 

2.00 

2.10 

Q3-Q1 

1.35 

0.85 

0.80 

N 

36 

36 

36 

Group 

D 

C 

B 

Note.  H  =  49.59, 

p  <  .001  (p  * 

.001  for  H 

>  16.27) 

Combined 

Boiler  Technician  and  Gunner's  Mate  Sample 

Statistic 

Number 

of  Callouts 

10 

27 

44 

LO  Score 

2.90 

2.30 

2.50 

Q1 

3.60 

2.70 

3.00 

Q2  (Median) 

3.90 

3.20 

3.60 

Q3 

4.15 

3.95 

4.30 

HI  Score 

6.00 

4.10 

5.20 

HI-LO 

3.10 

1.80 

2.70 

Q3-Q1 

0.55 

1.25 

1.30 

N 

15 

15 

14 

Group 

D 

C 

B 

Note.  H  -  22.17, 

p  <  .001  (p  ■ 

.001  for  H 

>  16.27) 
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62 


2.20 
3.55 
3.90 
4.30 
6.  OC 


62 


3.20 

4.55 

5.00 

6.00 

7.9C 

4.7C 

1.45 

15 

A 


Table  C-33 


Test  Item  Distribution  Statistics 
Task  3  (Tell  Nomenclature  of  Marked  Part) 
EXPLODED  VIEW:  SEQ/NC/E 


Sonar  Technician  Sample 


Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

1.60 

2.00 

2.00 

2.60 

Q1 

2.70 

3.05 

2.30 

3.40 

Q2  (Median) 

3.00 

3.65 

3.00 

4.00 

Q3 

3.20 

4.00 

3.30 

4.50 

HI  Score 

4.60 

5.00 

4.90 

10.50 

HI-LO 

3.00 

3.00 

2.90 

7.90 

Q3-Q1 

0.50 

0.95 

1.00 

1.10 

N 

36 

36 

36 

36 

Group 

A 

D 

C 

B 

Note.  H  =  41.64, 

p  <  .001  (p  = 

.001  for  H 

>  16.27) 

Combined 

Boiler  Technician 

and  Gunner's 

Mate  Sample 

Statistic 

Number  of  Callouts 

10 

27 

44 

62 

LO  Score 

2.20 

2.80 

3.00 

3.60 

Q1 

3.10 

3.85 

4.00 

4.20 

Q2  (Median) 

4.00 

4.20 

4.10 

4.70 

Q3 

5.55 

5.60 

5.05 

5.20 

HI  Score 

7.10 

7.80 

8.30 

7.90 

HI-LO 

4.90 

5.00 

5.30 

4.30 

Q3-Q1 

2.45 

1.75 

1.05 

1.00 

N 

15 

15 

15 

14 

Group 

A 

D 

C 

B 

Note.  H  =  5.51, 

p  >  .05  (p  =  .05 

for  H  >  7.82) 
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Table  C-34 


Test  Item  Distribution  Statistics 

Task  3  (Tell  Nomenclature  of  Marked  Part) 

EXPLODED  VIEW:  SEQ/NC/NE 

Sonar  Technician  Sample 

• 

•  _ . _ _ _  ■ 

1 

Statistic 

j  __ 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

1.50 

2.30 

1.80 

1.80 

Q1 

2.15 

3.20 

3.10 

2.60 

02  (Median) 

2.85 

3.85 

3.60 

3.00 

!  Q3 

3.20 

4.90 

4.10 

3.45 

HI  Score 

1 

4.80 

7.90 

5.50 

7.10 

HI-LO 

3.30 

5.60 

3.70 

5.30 

Q3-Q1 

1.05 

1.70 

1.00 

0.85 

i  N 

36 

36 

36 

36 

i  Group 

i 

B 

A 

D 

C 

1 

Note.  H  =  35.31, 

p  <  .001  (p  = 

.001  for  H 

>  16.27) 

; 

Combined 

Boiler  Technician  and  Gunner's  Mate  Sample 

Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

2.10 

2.50 

3.80 

3.00 

Q1 

2.60 

3.90 

4.05 

3.85 

02  (Median) 

3.15 

4.90 

4.30 

4.10 

Q3 

4.00 

7.35 

5.45 

4.75 

HI  Score 

4.30 

8.70 

9.20 

6.70 

HI-LO 

2.20 

6.20 

5.40 

3.70 

Q3-Q1 

1.40 

3.45 

1.40 

0.90 

N 

14 

15 

15 

15 

Group 

B 

A 

D 

C 

Note.  H  ■  16.88, 

p  <  .001  (p  * 

.001  for  H 

>  16.27) 

C-34 

Table  C-35 


Test  Item  Distribution  Statistics 
Task  3  (Tell  Nomenclature  of  Marked  Part) 
EXPLODED  VIEW:  RAN/C/E 


Sonar  Technician  Sample 


Statistic 

Number  of 

Callouts 

10 

27 

44  62 

LO  Score 

1.30 

2.50 

2.20  2.40 

Q1 

1.85 

3.20 

3.10  3.45 

Q2  (Median) 

2.20 

3.85 

3.65  4.05 

Q3 

2.80 

4.00 

4.15  4.55 

HI  Score 

3.20 

5.80 

5.60  11.50 

HI-LO 

1.90 

3.30 

3.40  9.10 

Q3-Q1 

0.95 

0.80 

1.05  1.10 

N 

36 

36 

36  36 

Group 

B 

A 

D  C 

Note.  H  =  68.95 

,  p  <  .001  (p  * 

.001  for  H  > 

16.27) 

Combined 

Boiler  Technician 

and  Gunner 

'  s  Mate  Sample 

Statistic 

Number  of 

Callouts 

10 

27 

44  62 

LO  Score 

1.70 

3.10 

3.20  4.00 

Q1 

2.50 

3.50 

4.40  4.55 

02  (Median) 

2.85 

4.80 

4.90  6.20 

Q3 

3.00 

5.75 

5.60  12.90 

HI  Score 

4.90 

7.60 

7.00  19.70 

HI-LO 

3.20 

4.50 

3.80  15.70 

Q3-Q1 

0.50 

2.25 

1.20  8.35 

N 

14 

15 

15  15 

Group 

B 

A 

D  C 

Note.  H  *  30.66, 

p  <  .001  (p  ■  .001  for  H  > 

16.27) 

--i.-'W.  . 


Table  C-36 

Test  Item  Distribution  Statistics 
Task  3  (Tell  Nomenclature  of  Marked  Part) 
EXPLODED  VIEW:  RAN/C/NE 


Sonar  Technician  Sample 


Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

1.70 

2.00 

1.80 

1.70 

Q1 

2.00 

2.50 

3.00 

3.05 

02  (Median) 

2.45 

3.00 

3.75 

3.60 

Q3 

3.00 

3.20 

4.10 

4.25 

HI  Score 

5.00 

10.10 

5.30 

7.20 

HI-LO 

3.30 

8.10 

3.50 

5.50 

Q3-Q1 

1.00 

0.70 

1.10 

1.20 

N 

36 

36 

36 

36 

Group 

A 

D 

C 

B 

Note.  H  *  AO. 36, 

p  <  .001  (p  * 

.001  for  H 

>  16.27) 

Combined 

Boiler  Technician 

and  Gunner's  Mate  Sample 

Statistic 

Number 

of  Callouts 

10 

27 

44 

62 

LO  Score 

2.20 

3.00 

3.20 

3.00 

Q1 

2.25 

3.45 

3.80 

3.80 

02  (Median) 

3.00 

4.10 

4.40 

4.05 

Q3 

3.75 

5.05 

5.15 

5.20 

HI  Score 

6.00 

8.10 

6.10 

7.30 

HI-LO 

3.80 

5.10 

2.90 

4.30 

Q3-Q1 

1.50 

1.60 

1.35 

1.40 

N 

15 

15 

15 

14 

Group 

A 

D 

C 

B 

Note.  H  *  15.88, 

p  <  .01  (p  ■  .01 

for  H  > 

11.34) 

C-3( 
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Table  C-37 


Test  Item  Distribution  Statistics 
Task  3  (Tell  Nomenclature  of  Marked  Part) 
EXPLODED  VIEW:  RAN/NC/E 


Sonar  Technician  Sample 


Statistic  Number  of  Callouts 


10  27  44  62 


LO  Score 

1.30 

1.30 

2.30  2.50 

Q1 

1.80 

2.60 

2.55  3.60 

Q2  (Median) 

2.15 

3.00 

3.65  4.00 

Q3 

2.80 

3.60 

4.00  4.55 

HI  Score 

4.00 

4.90 

5.60  6.00 

HI-LO 

2.70 

3.60 

3.30  3.50 

Q3-Q1 

1.00 

1.00 

1.45  0.95 

N 

36 

36 

36  36 

Group 

D 

C 

B  A 

Note.  H  *  63.6, 

p  <  .001  (p  = 

.001  for  H 

>  16.27) 

Combined  Boiler  Technician  and  Gunner's  Mate  Sample 


Statistic 


Number  of  Callouts 


10  27  44  62 


LO  Score 

1.30 

3.00 

3.20  2.80 

Q1 

2.30 

3.20 

4.00  4.90 

Q2  (Median) 

2.90 

3.50 

4.50  6.30 

Q3 

4.00 

4.10 

5.60  7.35 

HI  Score 

7.80 

6.50 

6.10  12.90 

HI-LO 

6.50 

3.50 

2.90  10.10 

Q3-Q1 

1.70 

0.90 

1.60  2.45 

N 

15 

15 

14  15 

Group 

D 

C 

B  A 

Note.  H  -  20.52, 

p  <  .001  (p  ■ 

.001  for  H 

>  16.27) 

Table  C-38 


Test  Item  Distribution  Statistics 
Task  3  (Tell  Nomenclature  of  Marked  Part) 
EXPLODED  VIEW:  RAN/NC/NE 


Sonar  Technician  Sample 


Statistic 

Number  of 

Callouts 

10 

27 

44 

62 

LO  Score 

1.80 

1.20 

1.80 

2.20 

Q1 

2.10 

2.00 

2.55 

Q2  (Median) 

2.85 

2.50 

msm 

Q3 

3.55 

3.15 

3.45 

HI  Score 

7.00 

4.60 

6.40 

13.70 

HI-LO 

5.20 

3.40 

11.50 

Q3-Q1 

1.45 

1.15 

1.25 

1.35 

N 

36 

36 

36 

36 

Group 

C 

B 

A 

D 

Note.  H  =  12.47, 

p  <  .01  (p  * 

.01  for  H  >  11 

.34) 

Combined 

Boiler  Technician 

and  Gunner' 

s  Mate  Sample 

Statistic 

Number  of 

Callouts 

10 

27 

44 

62 

LO  Score 

2.80 

2.70 

2.20 

3.10 

Q1 

3.00 

2.90 

3.05 

3.55 

Q2  (Median) 

3.40 

3.10 

3.90 

4.00 

Q3 

4.65 

4.10 

4.80 

4.50 

HI  Score 

11.70 

4.70 

7.10 

6.80 

HI-LO 

8.90 

2.00 

4.90 

3.70 

Q3-Q1 

1.65 

1.20 

1.75 

0.95 

N 

15 

14 

15 

15 

Group 

C 

B 

A 

D 

Note.  H  *  5.08,  p  >  .05  (p  *  .05  for  H  >  7.82) 


__  ^  _ _ 
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Table  C-39 


Test  Item  Distribution  Statistics 
Task  4  (Use  Zone  System  to  Point  to  Part  Given  Callout  Number) 
Exploded  View — 62  Callouts 

Sonar  Technician  Sample 


Statistic 

RAN/C/E 

RAN/C/NE 

RAN/NC/E 

RAN/NC/NE 

LO  Score 

5.10 

5.50 

2.70 

6.50 

Q1 

9.45 

9.05 

9.35 

8.40 

Q2  (Median) 

11.90 

12.15 

12.60 

11.16 

Q3 

18.90 

16.50 

16.60 

12.50 

HI  Score 

39.20 

23.80 

44.10 

37.60 

HI-LO 

34.10 

18.30 

41.40 

31.10 

Q3-Q1 

9.45 

7.45 

7.25 

4.10 

N 

36 

36 

36 

36 

Group 

A 

B 

C 

D 

Note.  H  *  5.07,  p  > 

.05  (p  -  .05 

for  H  >  7.82) 

Combined 

Boiler  Technician 

and  Gunner's 

Mate  Sample 

Statistic 

RAN/C/E 

RAN/C/NE 

RAN/NC/E 

RAN/NC/NE 

LO  Score 

3.80 

9.90 

4.80 

9.10 

Q1 

17.90 

14.00 

9.20 

12.70 

Q2  (Median) 

22.20 

16.55 

19.30 

17.60 

Q3 

35.70 

22.70 

34.80 

24.90 

HI  Score 

39.10 

44.80 

55.20 

48.10 

HI-LO 

35.30 

34.90 

50.40 

39.00 

Q3-Q1 

17.80 

8.70 

25.60 

12.20 

N 

15 

14 

15 

15 

Group 

A 

B 

C 

D 

Note.  H  -  1.12,  p  >  .05  (p  ■  .05  for  H  >  7.82) 
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Table  C-40 


Test  Item  Distribution  Statistics 
Task  5  (Use  Zone  System  to  Point  to  Part  Given  Nomenclature) 
Exploded  View — 62  Callouts 

Sonar  Technician  Sample 


Statistic 

RAN/C/E 

RAN/C/NE 

RAN/NC/E 

RAN/NC/NE 

LO  Score 

1.80 

.50 

1.60 

3.60 

Q1 

7.60 

4.70 

8.00 

11.35 

Q2  (Median) 

11.50 

14.25 

13.10 

15.95 

Q3 

17.40 

18.45 

21.30 

23.70 

HI  Score 

56.60 

45.20 

81.10 

62.40 

HI-LO 

54.80 

44.70 

79.50 

58.80 

Q3-Q1 

9.80 

13.75 

13.30 

12.35 

N 

36 

36 

36 

36 

Group 

D 

C 

B 

A 

Note.  H  *  5.40,  p  > 

.05  (p  -  .05 

for  H  >  7.82) 

Combined 

Boiler  Technician  and  Gunner's 

Mate  Sample 

Statistic 

RAN/C/E 

RAN/C/NE 

RAN/NC/E 

RAN/NC/NE 

L0  Score 

5.10 

6.90 

7.00 

4.60 

Q1 

8.60 

19.00 

13.30 

14.50 

Q2  (Median) 

20.80 

26.50 

16.90 

21.00 

Q3 

27.40 

34.10 

28.50 

44.25 

HI  Score 

68.70 

50.80 

51.30 

67.80 

HI-LO 

63.60 

43.90 

44.30 

63.20 

Q3-Q1 

18.80 

15.10 

15.20 

29.75 

N 

15 

15 

14 

15 

Group 

D 

C 

B 

A 

Note.  H  -  2.44,  p  >  .05  (p  ■  .05  for  H  >  7.82) 


APPENDIX  D 

EXAMPLES  OF  DRAWINGS  ON  WHICH 
UNEXPLAINED  PERCEPTUAL  PHENOMENA  OCCURRED 
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Figure  D-l.  Example  of  cross-sectional 
drawing  in  wh.ch  unexplained  perceptual 
phenomenon  operates. 


D-l/2 


Parts  Location  Index 


CALLOUT 

REF 

NUMBER 

DFSIG 

NAME 

1 

MP1 

Spring 

2 

MP31 

Thrust  Ball  Bearing 

3 

MP6 

Ball  Adapter 

4 

MP11 

Paper  Measuring  Device 

5 

MP25 

Gear 

6 

MP4 

Printing  Drum 

7 

MP20 

Flanged  Ball  Bearing 

8 

LI 

Electrical  Solenoid 

9 

MP8 

Knob 

10 

MP30 

Hub  Assembly  Gear 

11 

MP10 

Compression  Spring 

12 

MP28 

Flanged  Ball  Bearing 

13 

MP17 

Leveling  Roller 

14 

MP13 

Dial 

15 

MP29 

Positive  Shaft 

16 

MP32 

Hub  Assembly  Shaft 

27 

MP7 

Tension  Roller 

18 

R1 

Resistor 

19 

MP16 

Stud  Spring 

20 

MP2 

Scriber 

21 

MP23 

Timing  Gear 

22 

J2 

Switch  Assembly 

23 

MP22 

Block  Chain 

24 

MP21 

Output  Roller 

25 

MP36 

Coupl ing 

26 

MP19 

Roller  Shaft  Gear 

27 

MP34 

Sprocket  Wheel 

28 

S2 

Pushbutton  Switch 

29 

MP26 

Sprocket  Wheel 

30 

MP12 

Tension  Spring 

31 

MP27 

Position  Shaft 

32 

MP18 

Flanged  Ball  Bearing 

33 

MP15 

Bearing 

34 

MP40 

Scriber  Bracket 

35 

MP3 

Annular  Bearing 

36 

MP39 

Flanged  Ball  Bearing 

37 

MP9 

Instruction  Knob 

38 

MP37 

Adjustment  Bearing 

39 

MP24 

Annular  Bearing 

40 

MP14 

Shaft 

41 

MP33 

Cap  Assembly 

42 

MP3  5 

Spring 

43 

MP38 

Thrustball  Bearing 

44 

MP6 

Ball 

Figure  D-2a.  Example  of  exploded  view 
drawing  in  which  unexplained  perceptual 
phenomenon  operates  (44  callouts, 
target  item:  28). 


55 

MP54 

Oblique  1 

56 

MP39 

Flanged  1 

57 

MP37 

Aajustmei 

56 

MP51 

Annular  1 

55 

MP4 

PHntinq 

60 

MP40 

Scriber  { 

Parts  Location  Index 
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Oblique  Roller  Bracket 
Flanged  Ball  Bearing 
Adjustment  Bearing 
Annular  Bearing 
Printinq  Drum 
Scriber  Bracket 


CALLOUT 

REF 

NUMBER 

DESIG 

1 

MP  1 

2 

MP31 

3 

MP5 

4 

MP12 

5 

MP24 

6 

MP49 

7 

MP15 

8 

LI 

9 

J2 

10 

MP38 

11 

MP45 

12 

MP26 

13 

MP34 

14 

MP8 

15 

MP30 

16 

MP56 

17 

S2 

18 

MP4  2 

19 

R1 

20 

MP18 

21 

MP3 

22 

MP43 

23 

MP21 

24 

MP28 

25 

MP10 

26 

MP19 

27 

MP7 

28 

MP57 

29 

MP9 

30 

MP23 

31 

MP46 

32 

MP11 

33 

MP36 

34 

MP2 

35 

MP32 

36 

MP27 

37 

MP  14 

38 

MP16 

39 

MP17 

40 

MP22 

41 

MP55 

42 

MP47 

43 

MP25 

44 

MP13 

45 

MP33 

46 

MP35 

47 

MP41 

48 

MP53 

49 

MP44 

50 

MP52 

51 

MP20 

52 

MP6 

53 

MP50 

54 

MP29 

MP 1  Spring 

MP31  Thrust  Ball  Bearing 

MP5  Ball  Adapter 

MP12  Tension  Spring 

MP24  Annular  Bearing 

MP49  Set  Screw 

MP15  Bearing 

LI  Electrical  Solenoid 

J2  Switch  Assembly 

MP38  Thrustball  Bearing 

MP45  Roller  End  Bracket 

MP26  Sprocket  Wheel 

MP34  Sprocket  Wheel 

MP8  Knob 

MP30  Hub  Assembly  Gear 

MP56  Knob  Shaft 

S2  Pushbutton  Switch 

MP4 2  Stud  Clip 

R1  Resistor 

MP18  Flanged  Ball  Bearing 

MP3  Annular  Bearing 

MP43  Shouldered  Stud 

MP21  Output  Roller 

MP28  Flanged  Ball  Bearing 

MP10  Compression  Spring 

MP19  Roller  Shaft  Gear 

MP7  Tension  Roller 

MP57  Roller  Carriage 

MP9  Instruction  Knob 

MP23  Timing  Gear 

MP46  Roller  Mounting  Plate 

MP11  Paper  Measuring  Device 

MP36  Coupling 

MP2  Scriber 

MP32  Hub  Assembly  Shaft 

MP27  Position  Shaft 

MP 14  Shaft 

MP16  Stud  Spring 

MP 17  Leveling  Roller 

MP22  Block  Chain 

MP55  Roller  Mount 

MP47  Set  Screw 

MP25  Gear 

MP13  Dial 

MP33  Cap  Assembly 

MP35  Spring 

MP41  Roller  End  Bracket 

MP53  Bracket  Plate 

MP44  Switch  Bolt 

MP5 2  Shaft  Bracket 

MP20  Flanged  Ball  Bearing 

MP6  Ball 

MP50  Corner  Brace 

MP29  Positive  Shaft 

Figure  D-2b.  Example  of  exploded  view 
drawing  in  which  unexplained  perceptual 
phenomenon  operates  (62  callouts, 
target  item:  53).  D-5/6 
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